CN111459437A - Control method, controlled end, control end and system of remote screen - Google Patents
Control method, controlled end, control end and system of remote screen Download PDFInfo
- Publication number
- CN111459437A CN111459437A CN202010249955.6A CN202010249955A CN111459437A CN 111459437 A CN111459437 A CN 111459437A CN 202010249955 A CN202010249955 A CN 202010249955A CN 111459437 A CN111459437 A CN 111459437A
- Authority
- CN
- China
- Prior art keywords
- control
- screen
- network server
- controlled end
- controlled
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000004891 communication Methods 0.000 claims description 32
- 230000000977 initiatory effect Effects 0.000 claims description 23
- 230000001960 triggered effect Effects 0.000 claims description 10
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 10
- 238000004088 simulation Methods 0.000 description 4
- 241000208818 Helianthus Species 0.000 description 1
- 235000003222 Helianthus annuus Nutrition 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/0486—Drag-and-drop
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/547—Remote procedure calls [RPC]; Web services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/0876—Network architectures or network communication protocols for network security for authentication of entities based on the identity of the terminal or configuration, e.g. MAC address, hardware or software configuration or device fingerprint
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/1066—Session management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/70—Media network packetisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/161—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
- H04L69/162—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields involving adaptations of sockets based mechanisms
Abstract
The invention discloses a control method of a remote screen, a controlled end, a control end and a system, wherein the method only needs to install a client application program on the controlled end, the control end accesses a network server through a browser, the remote screen control of the controlled end can be realized, the control end does not need to install the client application program, the operation steps are greatly simplified, the method can complete the remote control without root of the controlled end, the operation steps are further simplified, the control efficiency of the remote screen is improved, the purpose of remotely assisting a user of the controlled end to operate the controlled end is achieved, and the method is beneficial for disabled to use mobile terminals such as mobile phones.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to a control method, a controlled end, a control end and a system of a remote screen.
Background
The remote screen is a technology for controlling a controlled end (far end) computer by using a control end (local end) computer, and the principle is that the screen of the controlled end computer is redirected to the control end computer, so that the screen of the controlled end computer can be equivalently operated by operating a virtual screen of the control end computer. The remote screen is widely applied to remote enterprise conferences, remote mobile offices, remote technical instruction and the like.
At present, famous Windows remote screen control software at home and abroad includes TeamViewer, VNC, sunflower and the like, and no matter which remote screen control software is used, on one hand, a controlled end and a control end both need to install a specified client application program, so that the operation is complicated; on the other hand, the controlled end root needs to be authorized (with root authority), which results in the data cleaning of the controlled end, and if the roots are batched, one controlled end needs to be operated, which is time-consuming and tedious in operation.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a control method, a controlled end, a control end and a system of a remote screen, and aims to solve the problems that the controlled end and the control end are required to install client programs and the like in the traditional method.
The invention solves the technical problems through the following technical scheme: a method of controlling a remote screen, comprising:
step 1: acquiring original video data of a controlled end screen, encoding and compressing the original video data into a video stream, and sending the video stream to a network server;
step 2: the control terminal reads the video stream from the network server through the access webpage, decodes and plays the video stream, and realizes real-time display of the screen of the controlled terminal on the access webpage of the control terminal;
and step 3: executing corresponding operation on a controlled end screen displayed in real time by a webpage, and sending a triggered operation instruction to a network server;
and 4, step 4: the controlled end receives the operation instruction transmitted from the network server and simulates the operation instruction to operate on a screen of the controlled end;
and 5: and (3) acquiring the controlled end screen in the operation process and after operation in real time on the control end access webpage through the screen real-time display of the steps 1 and 2, namely realizing the remote operation of the controlled end screen on the control end access webpage.
According to the control method, the communication connection is established between the controlled end and the network server, only the controlled end needs to install the client application program, the control end only needs to access the network server through the browser, the remote operation and control of the screen of the controlled end by the control end can be achieved, the control end does not need to install the client application program, the operation steps are greatly simplified, remote control can be completed by the method without root of the controlled end, the operation steps are further simplified, the control efficiency of the remote screen is improved, and the purpose of remotely assisting a user of the controlled end to operate the controlled end is achieved.
Further, the controlled end and the network server are in communication connection through a WebSocket protocol; and the network server is in communication connection with the control end through a WebSocket protocol. The WebSocket protocol is a network protocol based on TCP, and the protocol realizes full duplex communication between a controlled end and a server and between a control end browser and the server, can better save server resources and bandwidth and achieves the purpose of real-time communication.
Further, before step 1, a control connection between the controlled end and the control end is established, where the specific establishment method of the control connection is as follows:
the control end initiates a remote control request to the controlled end;
after the controlled terminal receives and allows the remote control request, a screen recording request is initiated at the controlled terminal, and the controlled terminal allows the screen recording request;
and initiating an authority opening request at the controlled end, wherein the controlled end allows the authority opening request.
Further, in step 3, the operation instruction includes a click instruction or a drag instruction.
Further, the click command or the drag command is determined according to the following criteria: and if the pixel difference between the ending coordinate and the starting coordinate is not more than 10 pixels, the command is a click command, otherwise, the command is a drag command.
Further, the control terminal is a PC.
Furthermore, the controlled terminal is a mobile terminal with a screen recording API.
The invention also provides a controlled end of the remote screen, and the controlled end is in communication connection with the network server through a network protocol based on TCP; the network server is in communication connection with the control end through a network protocol based on TCP, and the controlled end comprises:
the identity authentication module is used for completing identity authentication of the controlled terminal on the network server;
the request processing and initiating module is used for receiving and processing a remote control request sent by the network server, initiating and processing a screen recording request and initiating and processing an authority opening request;
the video coding module is used for acquiring original video data of a controlled end screen after the authority is opened, and coding and compressing the original video data into video streams to be sent to the network server;
the instruction receiving module is used for receiving the operation instruction transferred from the network server;
and the screen control module is used for simulating the operation instruction to carry out operation control on the controlled end screen.
The invention also provides a control end of the remote screen, and the control end is in communication connection with the network server through a network protocol based on TCP; the network server is in communication connection with the controlled end through a network protocol based on TCP; the control end comprises:
the list acquisition module is used for acquiring all lists of the controlled terminals completing the identity authentication from the network server;
the request sending module is used for selecting the controlled end and sending a remote control request to the selected controlled end;
a video decoding module, configured to decode a video stream obtained from the network server;
the video playing module is used for playing the decoded video stream and realizing the real-time display of the controlled end screen on the control end access webpage;
and the instruction sending module is used for executing corresponding operation on the controlled end screen displayed in real time on the webpage and sending the triggered operation instruction to the network server.
The present invention also provides a control system of a remote screen, comprising: the system comprises a controlled end, a control end and a network server; the controlled end and the network server are in communication connection through a network protocol based on TCP; the network server is in communication connection with the control end through a network protocol based on TCP, and the controlled end comprises:
the identity authentication module is used for completing identity authentication of the controlled terminal on the network server;
the request processing and initiating module is used for receiving and processing a remote control request sent by the network server, initiating and processing a screen recording request and initiating and processing an authority opening request;
the video coding module is used for acquiring original video data of a controlled end screen after the authority is opened, and coding and compressing the original video data into video streams to be sent to the network server;
the instruction receiving module is used for receiving the operation instruction transferred from the network server;
the screen control module is used for simulating the operation instruction to carry out operation control on the controlled end screen;
the control end comprises:
the list acquisition module is used for acquiring all lists of the controlled terminals completing the identity authentication from the network server;
the request sending module is used for selecting the controlled end and sending a remote control request to the selected controlled end;
a video decoding module, configured to decode a video stream obtained from the network server;
the video playing module is used for playing the decoded video stream and realizing the real-time display of the controlled end screen on the control end access webpage;
and the instruction sending module is used for executing corresponding operation on the controlled end screen displayed in real time on the webpage and sending the triggered operation instruction to the network server.
Advantageous effects
Compared with the prior art, the remote screen control method, the controlled end, the control end and the system provided by the invention have the advantages that the controlled end is in communication connection with the network server, only the controlled end needs to install the client application program, the control end only needs to access the network server through the browser, the remote operation and control of the control end on the screen of the controlled end can be realized, the control end does not need to install the client application program, the operation steps are greatly simplified, the remote control can be completed without root of the controlled end, the operation steps are further simplified, the control efficiency of the remote screen is improved, the purpose of remotely assisting a user of the controlled end to operate the controlled end is achieved, and the remote screen control method is beneficial for disabled people to use mobile terminals such as mobile phones.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only one embodiment of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
FIG. 1 is a flow chart of a remote screen control method in an embodiment of the invention;
FIG. 8 is a functional block diagram of a remote screen control in an embodiment of the present invention;
FIG. 2 is a diagram of an IMC remote management module according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating a dialog box for controlling a screen recording according to an embodiment of the present invention;
FIG. 4 is a real-time display diagram of a console side screen on a console side according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating an "accessibility" dialog in an embodiment of the invention;
FIG. 6 is a diagram illustrating IMC right to open in an embodiment of the present invention;
FIG. 7 is a diagram illustrating the determination of IMC permission to open according to an embodiment of the present invention;
FIG. 9 is a real-time display of a controlled end screen on the controlling end after completion of a control connection in an embodiment of the present invention;
FIG. 10 is a diagram illustrating operations of a click command and a drag command in an embodiment of the present invention;
FIG. 11 is a diagram of a network architecture for a remote desktop control system in an embodiment of the present invention;
FIG. 12 is a control timing diagram of the remote desktop control system in an embodiment of the present invention;
the system comprises a mobile terminal 1 (controlled terminal), a mobile terminal 2 (controlled terminal), a network server 3, a firewall 4 and a PC 5 (control terminal).
Detailed Description
The technical solutions in the present invention are clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1:
as shown in fig. 1 and 2, in the control method of the remote screen, a controlled end and a web server are in communication connection through a WebSocket protocol; the method comprises the following steps that the network server is in communication connection with a control end through a WebSocket protocol, the controlled end is a mobile terminal with a screen recording API, the control end is a PC, and the specific method comprises the following steps:
1. the controlled end and the control end establish control connection through the network server
Downloading and installing a client application program at a controlled terminal, completing identity authentication (namely registration) on the client application program, storing all the controlled terminals completing the identity authentication on a network server, accessing a webpage through a browser at a control terminal to obtain a list of all the controlled terminals completing the identity authentication from the network server, finding the controlled terminal from the list when remote control is needed, initiating a remote control request to the controlled terminal, initiating a screen recording request (the controlled terminal has a screen recording API) at the controlled terminal after the controlled terminal receives and allows the remote control request, and allowing the screen recording request by the controlled terminal; and then, initiating an authority opening request at the controlled end, wherein the controlled end allows the authority opening request, namely, the control connection between the controlled end and the control end is established through the network server.
2. And acquiring original video data of a controlled end screen, encoding and compressing the original video data into a video stream, and sending the video stream to a network server.
After the control connection is established, the controlled end acquires original video data of a screen of the controlled end through a self-contained screen recording API, encodes and compresses the original video data into a video stream in an H264 format, and then sends the video stream to the network server so that the control end can acquire the video stream from the network server.
3. The control terminal accesses the webpage through the browser to read the video stream in the H264 format from the network server, decodes and plays the video stream, and the screen of the controlled terminal is displayed on the webpage accessed by the control terminal in real time.
4. And executing corresponding operation on a controlled end screen displayed in real time on the webpage, and sending a triggered operation instruction to the network server.
5. And the controlled end receives the operation instruction transmitted from the network server and simulates the operation instruction to operate on a screen of the controlled end.
The operation instruction comprises a click instruction or a dragging instruction, and the coordinate of the position of the click screen on the mobile phone screen displayed by the control end is set as (x，y) Then, the coordinates of the click position on the screen of the controlled end in the simulation operation are (x，y) (ii) a Dragging the dragging command from one position to another position on the mobile phone screen displayed on the control end, wherein the starting coordinates are (startX, startY) and the ending coordinates are (stopX, stopY), so that the starting coordinates (st) and the ending coordinates (st) on the controlled end screen are obtained during the simulation operationartX, startY) to coordinates (stopX, stopY). The click and the sliding can be simulated on the controlled end through the click instruction and the drag instruction, the control operation on the controlled end is realized, and then the screen of the controlled end is displayed on an access webpage of the control end in real time through the screen recording API, namely, the remote screen control and operation of the control end on the controlled end are realized.
The judgment basis of the click instruction and the drag instruction is as follows: and if the pixel difference between the ending coordinate and the starting coordinate is not more than 10 pixels, the command is a click command, otherwise, the command is a drag command.
6. And (3) acquiring the controlled end screen in the operation process and after operation in real time on the control end access webpage through the screen real-time display of the steps 1 and 2, namely realizing the remote operation of the controlled end screen on the control end access webpage.
Example 2
The remote screen Control method, the controlled terminal, the Control terminal and the Control System of the present invention are described by taking an XX Mobile device management System (IMC) as an example to implement remote screen Control. The XX mobile equipment management system is the prior art and can be accessed through a browser of a computer, and the specific access address is as follows: http:// cs. infogo. cn:9292/tp/login. html. The XX mobile equipment management system is a mobile terminal management platform developed based on an EMM framework, and is dedicated to equipment management, APP management, content management, remote management, network admission control and the like of a mobile terminal so as to meet the requirements of unit customers on mobile terminal security management, and the specific introduction can refer to http:// www.infogo.com.cn/inc.
In this embodiment, the controlled end is a mobile phone (with a screen recording api) with an android system 7.0 or more, the controlled end may also be an android PAD, the control end is a computer (or a PC), the control end may also be a mobile terminal such as a mobile phone and an IPAD, and the specific operation steps include:
1. and installing a client application program (APP) of the XX mobile equipment management system on the mobile phone, and completing the identity authentication of the mobile phone on the XX mobile equipment management system APP.
The mobile phone accesses the XX mobile equipment management system through the browser, downloads and installs the APP, and then completes registration on the AAP, namely the identity authentication of the mobile phone on the APP is completed, and the mobile phones completing the identity authentication are all displayed in a list of the XX mobile equipment management system, and the list is stored on a network server.
2. The XX mobile equipment management system is accessed on the computer through the browser, the mobile phone needing remote control is found from the list acquired by the network server, and the control connection between the mobile phone and the computer is established, so that the remote screen of the mobile phone can be controlled and operated.
The mobile phone is in communication connection with the network server through a WebSocket protocol, the network server is in communication connection with the computer through the WebSocket protocol, the WebSocket protocol realizes full duplex communication between the mobile phone and the network server and between the computer browser and the network server, and can better save server resources and bandwidth and achieve the purpose of real-time communication.
The specific establishment method of the control connection between the mobile phone and the computer comprises the following steps:
2.1 in the XX mobile device management system accessed by the computer browser, the controlled mobile phone is selected in the list, and then the remote control is clicked under the remote management module (as shown in fig. 3, namely the control end sends a remote control request to the controlled end).
2.2 popping up an allowance dialog box on the mobile phone (namely, the controlled terminal receives the remote control request and allows); after the remote control is allowed, a screen recording dialog box pops up on the mobile phone, and the screen recording dialog box is clicked to start immediately (as shown in fig. 4, namely, the controlled terminal initiates a screen recording request and allows the screen recording request), so that the real-time display of the mobile phone screen on the computer is realized (as shown in fig. 5).
2.3 after the screen is displayed in real time, popping up an auxiliary function dialog box (as shown in fig. 6, the auxiliary function is authorized) on the mobile phone, clicking the 'setting', opening the IMC authority (as shown in fig. 7), and then clicking the 'determination' (as shown in fig. 8, namely, the controlled end initiates the authority opening request and allows the authority opening request), namely, completing the control connection between the mobile phone and the computer.
After the control connection is established between the mobile phone and the computer, the operation can be carried out on the mobile phone screen displayed by the computer browser, namely, the remote screen control operation of the mobile phone is realized.
3. Calling a screen recording API (application program interface) by the APP to start screen recording, directly acquiring original video data of a mobile phone screen by adopting the screen recording API, encoding and compressing the YUV format data of the original video into an H264 format video stream, and sending the video stream to a network server through a WebSocket protocol.
4. The XX mobile device management system accessed by the control-side browser obtains the H264 format video stream from the network server through the WebSocket protocol, decodes and plays the video stream, and realizes real-time display of the mobile phone screen on the computer, as shown in fig. 5 and 9.
5. Corresponding operation is executed on a mobile phone screen displayed by a computer browser, and a triggered operation instruction is sent to a network server through a WebSocket protocol.
6. The mobile phone receives the operation instruction transmitted from the network server through the WebSocket protocol, and simulates the operation instruction to operate on the controlled end screen.
The operation instruction comprises a click instruction and a drag instruction, and the coordinates of the click position of the click instruction on the mobile phone screen displayed by the computer are (100 px, 500 px), so that the coordinates of the click position on the mobile phone screen during the simulation operation are (100 px, 500 px); the drag command is dragged from one location to another location on the mobile phone screen displayed by the computer, the start coordinate is (100 px, 800 px), the end coordinate is (500 px, 800 px), and then the command is slid from the coordinate (100 px, 800 px) to the coordinate (500 px, 800 px) on the mobile phone screen during the simulation operation, as shown in fig. 10. The click and the sliding can be simulated on the mobile phone through the click instruction and the drag instruction, the control operation on the mobile phone is realized, and then the screen of the mobile phone is displayed on the window of the computer in real time through the screen recording API, namely, the remote screen control and the operation of the computer on the mobile phone are realized.
7. And (4) real-time display is carried out through the screens in the steps 3 and 4, the mobile phone screen in the operation process and after operation is obtained in real time on the XX mobile device management system accessed by the computer browser, and then remote operation of the mobile phone screen is realized.
According to the control method, only the controlled end needs to install the APP, the control end only needs to access the network server through the browser, remote screen control over the controlled end can be achieved, the control end does not need to be installed, operation steps are greatly simplified, remote control can be achieved without root of the controlled end, the operation steps are further simplified, control efficiency of a remote screen is improved, and the purpose of remotely assisting a user of the controlled end to operate the controlled end is achieved.
Example 3
The invention also provides a controlled end of the remote screen, and the controlled end is in communication connection with the network server through a network protocol based on TCP; the network server is in communication connection with the control end through a network protocol based on TCP, and the controlled end comprises:
the identity authentication module is used for completing identity authentication of the controlled terminal on the network server;
the request processing and initiating module is used for receiving and processing a remote control request sent by the network server, initiating and processing a screen recording request and initiating and processing an authority opening request;
the video coding module is used for acquiring original video data of a controlled end screen after the authority is opened, and coding and compressing the original video data into video streams to be sent to the network server;
the instruction receiving module is used for receiving the operation instruction transferred from the network server;
and the screen control module is used for simulating the operation instruction to carry out operation control on the controlled end screen.
The invention also provides a control end of the remote screen, and the control end is in communication connection with the network server through a network protocol based on TCP; the network server is in communication connection with the controlled end through a network protocol based on TCP; the control end comprises:
the list acquisition module is used for acquiring all lists of the controlled terminals completing the identity authentication from the network server;
the request sending module is used for selecting the controlled end and sending a remote control request to the selected controlled end;
a video decoding module, configured to decode a video stream obtained from the network server;
the video playing module is used for playing the decoded video stream and realizing the real-time display of the controlled end screen on the control end access webpage;
and the instruction sending module is used for executing corresponding operation on the controlled end screen displayed in real time on the webpage and sending the triggered operation instruction to the network server.
As shown in fig. 11, the present invention also provides a control system of a remote screen, including: the system comprises a controlled end, a control end and a network server; the controlled end and the network server are in communication connection through a network protocol based on TCP; the network server is in communication connection with the control end through a network protocol based on TCP, and the controlled end comprises:
the identity authentication module is used for completing identity authentication of the controlled terminal on the network server;
the request processing and initiating module is used for receiving and processing a remote control request sent by the network server, initiating and processing a screen recording request and initiating and processing an authority opening request;
the video coding module is used for acquiring original video data of a controlled end screen after the authority is opened, and coding and compressing the original video data into video streams to be sent to the network server;
the instruction receiving module is used for receiving the operation instruction transferred from the network server;
the screen control module is used for simulating the operation instruction to carry out operation control on the controlled end screen;
the control end comprises:
the list acquisition module is used for acquiring all lists of the controlled terminals completing the identity authentication from the network server;
the request sending module is used for selecting the controlled end and sending a remote control request to the selected controlled end;
a video decoding module, configured to decode a video stream obtained from the network server;
the video playing module is used for playing the decoded video stream and realizing the real-time display of the controlled end screen on the control end access webpage;
and the instruction sending module is used for executing corresponding operation on the controlled end screen displayed in real time on the webpage and sending the triggered operation instruction to the network server.
As shown in fig. 12, the control sequence of the control system of the present invention is: after a controlled end (namely a client) passes identity authentication, an administrator selects the controlled end on a webpage (namely a management background) accessed by a control end and sends a remote control request, the remote control request is sent to a network server (namely a background WebSocket server) through a WebSocket protocol, the network server informs the controlled end to start remote control, the controlled end receives the remote control request and allows remote control, the controlled end starts a screen recording function of the controlled end, directly obtains original video data through a screen recording API (application program interface), codes the original video data into video streams and sends the video streams to the network server, and before the control end sends out a remote cancellation request, the controlled end continuously sends the video streams to the network server so that a screen of the controlled end is displayed in real time at the control end; after receiving the video stream from the network server, the control end decodes and plays the video stream; clicking or dragging a mouse in a controlled end screen of a control end, transmitting a clicking or dragging instruction to a network server, forwarding the network server to the controlled end, simulating a clicking or sliding screen on the controlled end screen, continuously sending a video stream to the network server by the controlled end, sending the video stream to the control end by the network server, decoding and playing on a webpage accessed by a browser of the control end, namely acquiring the controlled end screen in the operation process and after the operation in real time on the webpage accessed by the control end, and realizing the remote operation of the controlled end screen on the webpage accessed by the control end.
The above disclosure is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of changes or modifications within the technical scope of the present invention, and shall be covered by the scope of the present invention.
Claims (10)
1. A method for controlling a remote screen, comprising:
step 1: acquiring original video data of a controlled end screen, encoding and compressing the original video data into a video stream, and sending the video stream to a network server;
step 2: the control terminal reads the video stream from the network server through the access webpage, decodes and plays the video stream, and realizes real-time display of the screen of the controlled terminal on the access webpage of the control terminal;
and step 3: executing corresponding operation on a controlled end screen displayed in real time by a webpage, and sending a triggered operation instruction to a network server;
and 4, step 4: the controlled end receives the operation instruction transmitted from the network server and simulates the operation instruction to operate on a screen of the controlled end;
and 5: and (3) acquiring the controlled end screen in the operation process and after operation in real time on the control end access webpage through the screen real-time display of the steps 1 and 2, namely realizing the remote operation of the controlled end screen on the control end access webpage.
2. The control method of a remote screen according to claim 1, wherein: the controlled end and the network server are in communication connection through a WebSocket protocol; and the network server is in communication connection with the control end through a WebSocket protocol.
3. A control method of a remote screen as claimed in claim 1 or 2, wherein: before step 1, a control connection between the controlled end and the control end is established, and the specific establishment method of the control connection is as follows:
the control end initiates a remote control request to the controlled end;
after the controlled terminal receives and allows the remote control request, a screen recording request is initiated at the controlled terminal, and the controlled terminal allows the screen recording request;
and initiating an authority opening request at the controlled end, wherein the controlled end allows the authority opening request.
4. A control method of a remote screen according to claim 3, wherein: in the step 3, the operation instruction includes a click instruction or a drag instruction.
5. The control method of a remote screen according to claim 4, wherein: the judgment basis of the click instruction or the drag instruction is as follows: and if the pixel difference between the ending coordinate and the starting coordinate is not more than 10 pixels, the command is a click command, otherwise, the command is a drag command.
6. The control method of a remote screen according to claim 1, wherein: the control end is a PC.
7. The control method of a remote screen according to claim 1, wherein: the controlled terminal is a mobile terminal with a screen recording API.
8. A controlled end of a remote screen, comprising: the controlled end and the network server are in communication connection through a network protocol based on TCP; the network server is in communication connection with the control end through a network protocol based on TCP, and the controlled end comprises:
the identity authentication module is used for completing identity authentication of the controlled terminal on the network server;
the request processing and initiating module is used for receiving and processing a remote control request sent by the network server, initiating and processing a screen recording request and initiating and processing an authority opening request;
the video coding module is used for acquiring original video data of a controlled end screen after the authority is opened, and coding and compressing the original video data into video streams to be sent to the network server;
the instruction receiving module is used for receiving the operation instruction transferred from the network server;
and the screen control module is used for simulating the operation instruction to carry out operation control on the controlled end screen.
9. A control end of a remote screen is characterized in that: the control end is in communication connection with the network server through a network protocol based on TCP; the network server is in communication connection with the controlled end through a network protocol based on TCP; the control end comprises:
the list acquisition module is used for acquiring all lists of the controlled terminals completing the identity authentication from the network server;
the request sending module is used for selecting the controlled end and sending a remote control request to the selected controlled end;
a video decoding module, configured to decode a video stream obtained from the network server;
the video playing module is used for playing the decoded video stream and realizing the real-time display of the controlled end screen on the control end access webpage;
and the instruction sending module is used for executing corresponding operation on the controlled end screen displayed in real time on the webpage and sending the triggered operation instruction to the network server.
10. A control system for a remote screen, comprising: the system comprises a controlled end, a control end and a network server; the controlled end and the network server are in communication connection through a network protocol based on TCP; the network server is in communication connection with the control end through a network protocol based on TCP, and the controlled end comprises:
the identity authentication module is used for completing identity authentication of the controlled terminal on the network server;
the request processing and initiating module is used for receiving and processing a remote control request sent by the network server, initiating and processing a screen recording request and initiating and processing an authority opening request;
the video coding module is used for acquiring original video data of a controlled end screen after the authority is opened, and coding and compressing the original video data into video streams to be sent to the network server;
the instruction receiving module is used for receiving the operation instruction transferred from the network server;
the screen control module is used for simulating the operation instruction to carry out operation control on the controlled end screen;
the control end comprises:
the list acquisition module is used for acquiring all lists of the controlled terminals completing the identity authentication from the network server;
the request sending module is used for selecting the controlled end and sending a remote control request to the selected controlled end;
a video decoding module, configured to decode a video stream obtained from the network server;
the video playing module is used for playing the decoded video stream and realizing the real-time display of the controlled end screen on the control end access webpage;
and the instruction sending module is used for executing corresponding operation on the controlled end screen displayed in real time on the webpage and sending the triggered operation instruction to the network server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010249955.6A CN111459437A (en) | 2020-04-01 | 2020-04-01 | Control method, controlled end, control end and system of remote screen |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010249955.6A CN111459437A (en) | 2020-04-01 | 2020-04-01 | Control method, controlled end, control end and system of remote screen |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111459437A true CN111459437A (en) | 2020-07-28 |
Family
ID=71684285
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010249955.6A Pending CN111459437A (en) | 2020-04-01 | 2020-04-01 | Control method, controlled end, control end and system of remote screen |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111459437A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112543348A (en) * | 2020-11-27 | 2021-03-23 | 深圳市慧为智能科技股份有限公司 | Remote screen recording method, device, equipment and computer readable storage medium |
| CN113433860A (en) * | 2021-06-25 | 2021-09-24 | 山东齐鲁数通科技有限公司 | Desktop remote control method and system |
| CN113676556A (en) * | 2021-09-29 | 2021-11-19 | 山东云缦智能科技有限公司 | Screen control system and method based on WebSocket protocol |
| CN115103220A (en) * | 2022-07-15 | 2022-09-23 | 福建合诚信息科技有限公司 | Remote control method for browser screen recording |
| CN117112104A (en) * | 2023-08-24 | 2023-11-24 | 浙江远算科技有限公司 | Local storage mapping method, equipment and medium based on remote desktop gateway |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103414751A (en) * | 2013-07-16 | 2013-11-27 | 广东工业大学 | PC screen content sharing/interaction control method |
| CN104219314A (en) * | 2014-09-10 | 2014-12-17 | 珠海市君天电子科技有限公司 | Electronic device and remote control method and system thereof |
| WO2015117367A1 (en) * | 2014-08-21 | 2015-08-13 | 中兴通讯股份有限公司 | Remote assistance control method and device |
| CN105807651A (en) * | 2016-03-03 | 2016-07-27 | 腾讯科技（深圳）有限公司 | Remote terminal controlling method, apparatus and system |
| CN107483576A (en) * | 2017-08-12 | 2017-12-15 | 厦门多米诺数据科技有限公司 | A kind of system and method that remote control is carried out to Android APP |
-
2020
- 2020-04-01 CN CN202010249955.6A patent/CN111459437A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103414751A (en) * | 2013-07-16 | 2013-11-27 | 广东工业大学 | PC screen content sharing/interaction control method |
| WO2015117367A1 (en) * | 2014-08-21 | 2015-08-13 | 中兴通讯股份有限公司 | Remote assistance control method and device |
| CN104219314A (en) * | 2014-09-10 | 2014-12-17 | 珠海市君天电子科技有限公司 | Electronic device and remote control method and system thereof |
| CN105807651A (en) * | 2016-03-03 | 2016-07-27 | 腾讯科技（深圳）有限公司 | Remote terminal controlling method, apparatus and system |
| CN107483576A (en) * | 2017-08-12 | 2017-12-15 | 厦门多米诺数据科技有限公司 | A kind of system and method that remote control is carried out to Android APP |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112543348A (en) * | 2020-11-27 | 2021-03-23 | 深圳市慧为智能科技股份有限公司 | Remote screen recording method, device, equipment and computer readable storage medium |
| CN113433860A (en) * | 2021-06-25 | 2021-09-24 | 山东齐鲁数通科技有限公司 | Desktop remote control method and system |
| CN113676556A (en) * | 2021-09-29 | 2021-11-19 | 山东云缦智能科技有限公司 | Screen control system and method based on WebSocket protocol |
| CN115103220A (en) * | 2022-07-15 | 2022-09-23 | 福建合诚信息科技有限公司 | Remote control method for browser screen recording |
| CN117112104A (en) * | 2023-08-24 | 2023-11-24 | 浙江远算科技有限公司 | Local storage mapping method, equipment and medium based on remote desktop gateway |
| CN117112104B (en) * | 2023-08-24 | 2024-03-29 | 浙江远算科技有限公司 | Local storage mapping method, equipment and medium based on remote desktop gateway |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111459437A (en) | Control method, controlled end, control end and system of remote screen | |
| CN102833323B (en) | Method and system for remote control of controlled terminal by server | |
| US7844442B2 (en) | System and method for providing a remote user interface for an application executing on a computing device | |
| CN102739800A (en) | System and method for remotely controlling browser Web page based on internet | |
| US20190230165A1 (en) | Remote desktop sharing and collaboration via image scanning | |
| WO2011101246A1 (en) | Metadata capture for screen sharing | |
| CN105025022B (en) | A kind of remotely generation control method and system | |
| CN102761603A (en) | Webpage flash video redirection method in VDI environment | |
| CN103973787A (en) | Method and system for remotely controlling display terminals through mobile intelligent terminal | |
| CN103942044A (en) | Remote assistance method and system in application of mobile terminal | |
| CN110694267A (en) | Cloud game implementation method and device | |
| US20140243083A1 (en) | Apparatus and method of providing cloud service using game platform based on streaming | |
| WO2012151957A1 (en) | Server, client, and method and system for playing video file remotely therewith | |
| CN107256153A (en) | A kind of virtual machine RFB agreements desktop resolution ratio adaptively realizes system and method | |
| WO2024037110A1 (en) | Data processing method and apparatus, device, and medium | |
| CN108762659B (en) | Cloud terminal control method and device, storage medium and electronic device | |
| CN112449203A (en) | Game live broadcast method and device, electronic equipment and storage medium | |
| JP7101313B2 (en) | Providing virtual desktops within a computing environment | |
| US20070073843A1 (en) | Method and device for video projection | |
| JP2007512560A5 (en) | ||
| CN104007986A (en) | Cross-system remote real-time interaction technology | |
| CN101409731A (en) | Remote control method for household electrical equipment through IE | |
| CN112653787A (en) | Display method, display device, storage medium and electronic equipment | |
| CN107222783A (en) | A kind of video based on SPICE protocol redirects implementation method | |
| CN116233209A (en) | Remote management scheduling method for GPU cloud rendering service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200728 |
|
| RJ01 | Rejection of invention patent application after publication |