CN115877725A

CN115877725A - Remote control method and device

Info

Publication number: CN115877725A
Application number: CN202111154961.4A
Authority: CN
Inventors: 宁乐
Original assignee: Qingdao Haier Technology Co Ltd; Haier Smart Home Co Ltd
Current assignee: Qingdao Haier Technology Co Ltd; Haier Smart Home Co Ltd
Priority date: 2021-09-29
Filing date: 2021-09-29
Publication date: 2023-03-31

Abstract

The invention provides a remote control method and a remote control device, which comprise the following steps: after the first terminal shares the control function of the second terminal on the equipment, receiving a media stream transmitted by a server; playing the media stream on the shared page; and when the control operation aiming at the target function on the shared page is detected, generating a control instruction of the target function, and transmitting the control instruction to the target equipment through the server. In the invention, the server caches the screenshot transmitted by the second terminal to generate the media stream, so that the first terminal does not have image distortion and pause in image receiving of the media stream, and the function of the equipment which is required to be controlled by the user and corresponds to the second terminal can be identified through the media stream, so that the second terminal can accurately and timely control the equipment which is required to be controlled by the second terminal, and the reliability of remote control of household appliances is improved.

Description

Remote control method and device

Technical Field

The present invention relates to remote control technologies, and in particular, to a remote control method and apparatus.

Background

With the development of society and the development of household equipment technology, intelligent household appliances become indispensable appliances in people's lives.

Special crowds such as the old have certain difficulty in learning to use intelligent household appliances. In order to facilitate the use of intelligent household appliances by these special groups, control devices having a take-over function have been developed. The special crowd can start the takeover function in the control equipment, so that the control equipment can share the screen, and then the control page shared by the screen requests other people to remotely control the intelligent household appliance through the terminal.

At present, in the screen sharing technology, a requesting party and a requested party which need to take over are on line at the same time, that is, a terminal of the requesting party needs to transmit an image or a video to a terminal of the requested party in real time. However, when the network performance of the terminal is poor, the transmitted image or video will be jammed and distorted, which results in that the household appliance cannot be controlled and the reliability of remote control of the household appliance is poor.

Disclosure of Invention

The invention provides a remote control method and a remote control device, which are used for solving the problem of poor reliability of remote control of household appliances.

In one aspect, the present invention provides a remote control method applied to a first terminal, including:

after the first terminal shares a control function of a second terminal on equipment, receiving a media stream transmitted by a server, wherein the media stream is generated by the server according to a screenshot transmitted by the second terminal, and the screenshot is used for recording the operation of a user on a control page on the second terminal;

playing the media stream on a shared page, wherein the media stream comprises target equipment to be controlled by the second terminal and target functions to be controlled by the target equipment;

when the control operation aiming at the target function on the shared page is detected, generating a control instruction of the target function, and transmitting the control instruction to the target equipment through the server.

In one embodiment, the step of transmitting the control instruction to the target device through the server includes:

generating icon information corresponding to the control instruction;

and transmitting the icon information to the second terminal through the server, and transmitting the control instruction to the target device through the server, wherein the second terminal adds an icon corresponding to the icon information to the control page, and when the touch operation of the icon is detected, the second terminal sends the control instruction to the target device.

In an embodiment, before the step of receiving the media stream transmitted by the server after the first terminal shares the control function of the second terminal on the device, the method further includes:

receiving a takeover request sent by the second terminal;

acquiring an encrypted message according to the takeover request, and carrying out takeover verification according to the encrypted message;

and when the taking-over check is passed, sending a prompt message that the taking-over check is passed to the second terminal, wherein the prompt message that the taking-over check is passed is used for the second terminal to authorize the first terminal to share the control function of the second terminal.

In an embodiment, the step of receiving the media stream transmitted by the server includes:

when receiving prompt information which is sent by the server and generated by the media stream, stopping sending a picture displayed by the first terminal to the second terminal through the server, and sending a first acquisition request to the server;

and receiving a response of the first acquisition request sent by the server, wherein the response of the second acquisition request comprises the media stream.

In one embodiment, the step of receiving the image information transmitted by the server comprises:

acquiring a second acquisition request input through the shared page, wherein the second acquisition request is used for requesting to acquire the media stream;

sending the second acquisition request to the server;

and receiving a response of the second acquisition request sent by the server, wherein the response of the second acquisition request comprises the media stream.

In another aspect, the present invention provides a remote control method applied to a second terminal, including:

when the second terminal authorizes the first terminal to share the control function of the second terminal on the equipment, and detects the operation of a control page of the second terminal, generating a screenshot corresponding to the control page, wherein the screenshot is used for recording the operation of a user on the control page;

transmitting each screenshot to the server, wherein each screenshot is used for the server to generate a media stream, and the first terminal controls a target function of a target device according to the media stream sent by the server;

and receiving prompt information of target function control sent by target equipment controlled by the second terminal, wherein the prompt information of target function control is sent by the first terminal after the first terminal controls the target function of the target equipment.

In an embodiment, after the step of receiving the prompt information for target function control sent by the target device controlled by the second terminal, the method further includes:

receiving icon information sent by the server;

and adding an icon corresponding to the icon information on the control page, wherein the second terminal sends the control instruction to the target device when detecting the touch operation of the icon.

In an embodiment, before the step that the second terminal authorizes the first terminal to share the control function of the second terminal to the device, and when a user operates a control page of the second terminal, and generates a screenshot corresponding to the control page, the method further includes:

acquiring an encrypted message, and generating a take-over request according to the encrypted message;

sending the takeover request to the first terminal, wherein the encrypted message is used for the first terminal to carry out takeover verification;

and when receiving prompt information which is sent by the second terminal and passes the take-over check, sending authorization information to the second terminal, wherein the authorization information is used for the second terminal to authorize the first terminal to share the control function of the second terminal on the equipment.

In an embodiment, the step of transmitting each screenshot to the server includes:

outputting a selection interface, wherein the selection interface displays a sharing mode of the second terminal for sharing the control function;

acquiring selection information of the selection interface;

when the sharing mode is determined to be delayed sharing according to the selection information, generating sharing information corresponding to the delayed sharing;

and sending the sharing information to the server, wherein the sharing information is used for indicating that the sharing mode is delayed sharing.

In another aspect, the present invention provides a remote control method applied to a server, including:

after a first terminal shares a control function of a second terminal on equipment, receiving various screenshots transmitted by the second terminal, wherein the screenshots are used for recording the operation of a user on a control page on the second terminal;

generating a media stream according to each screenshot, and sending the media stream to the first terminal, wherein the media stream comprises target equipment to be controlled by the second terminal and target functions to be controlled by the target equipment;

and receiving a control instruction sent by the first terminal, and sending the control instruction to the target device, wherein the control instruction is generated when the second terminal detects a control operation for the target function on the shared page.

In an embodiment, the step of receiving the control instruction sent by the first terminal and sending the control instruction to the target device includes:

receiving a control instruction sent by the first terminal and icon information corresponding to the control instruction;

and sending the control instruction to the target equipment, and sending the icon information to the second terminal, wherein the second terminal adds an icon corresponding to the icon information to the control page, and sends the control instruction to the target equipment when detecting the touch operation of the icon.

In an embodiment, the step of sending the media stream to the first terminal includes:

the prompt information generated by the media stream is sent to the first terminal;

receiving a first acquisition request sent by the first terminal;

and sending a response of the first acquisition request to the first terminal, wherein the response of the first acquisition request comprises the media stream.

receiving a second acquisition request sent by the first terminal;

and sending a response of the second acquisition request to the first terminal, wherein the response of the second acquisition request comprises the media stream.

In an embodiment, the step of generating a media stream according to each screenshot includes:

receiving shared information sent by the second terminal;

and when determining that the first terminal delays to share the control function of the second terminal according to the sharing information, generating a media stream according to each screenshot.

In another aspect, the present invention provides a terminal, including:

the first receiving module is used for receiving a media stream transmitted by a server after the first terminal shares a control function of a second terminal on equipment, wherein the media stream is generated by the server according to a screenshot transmitted by the second terminal, and the screenshot is used for recording the operation of a user on a control page on the second terminal;

the playing module is used for playing the media stream on a shared page, wherein the media stream comprises target equipment to be controlled by the second terminal and target functions to be controlled by the target equipment;

the first generation module is used for generating a control instruction of the target function when detecting the control operation aiming at the target function on the shared page, and transmitting the control instruction to the target equipment through the server.

In another aspect, the present invention provides a terminal, including:

the second receiving module is used for generating a screenshot corresponding to a control page when the second terminal authorizes the first terminal to share the control function of the second terminal on the equipment and detects the operation of the control page aiming at the second terminal, wherein the screenshot is used for recording the operation of a user on the control page;

the transmission module is used for transmitting each screenshot to the server, wherein each screenshot is used for the server to generate a media stream, and the first terminal controls a target function of a target device according to the media stream sent by the server;

the second receiving module is further configured to receive a prompt message for target function control sent by a target device controlled by the second terminal, where the prompt message for target function control is sent by the first terminal after controlling the target function of the target device.

In another aspect, the present invention provides a server, comprising:

the third receiving module is used for receiving various screenshots transmitted by a second terminal after the first terminal shares the control function of the second terminal on the equipment, wherein the screenshots are used for recording the operation of a user on a control page on the second terminal;

a second generating module, configured to generate a media stream according to each screenshot, and send the media stream to the first terminal, where the media stream includes a target device to be controlled by the second terminal and a target function to be controlled by the target device;

the third receiving module is further configured to receive a control instruction sent by the first terminal, and send the control instruction to the target device, where the control instruction is generated when the second terminal detects a control operation for the target function on the shared page.

The remote control method and the remote control device provided by the invention have the advantages that after a first terminal shares a control function of a second terminal on equipment, a media stream generated by a server based on all screenshots of the operation of a user on a control page sent by the second terminal is received, the media stream is played on the shared page so that the user corresponding to the first terminal can identify target equipment needing to be controlled by the user of the second terminal and the control operation of the target equipment, a control instruction corresponding to the target function is generated when the control operation aiming at the target function of the target equipment on the shared page is detected, and the control instruction is sent to the target equipment through the server. In the invention, the server caches all screenshots transmitted by the second terminal to generate the media stream, so that the first terminal cannot have image distortion and blockage when receiving the media stream, and can identify the function of equipment which needs to be controlled by a user and corresponds to the second terminal through the media stream, so that the second terminal can accurately and timely control the equipment which needs to be controlled by the second terminal, and the reliability of remote control of household appliances is improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a system architecture diagram of the present invention implementing a remote control method;

FIG. 2 is a flowchart illustrating a remote control method according to a first embodiment of the present invention;

FIG. 3 is a flowchart illustrating a remote control method according to a second embodiment of the present invention;

FIG. 4 is a flowchart illustrating a remote control method according to a third embodiment of the present invention;

FIG. 5 is a functional block diagram of the terminal according to the present invention;

FIG. 6 is a diagram illustrating another functional module of the terminal according to the present invention;

FIG. 7 is a functional block diagram of a server according to the present invention;

fig. 8 is a schematic diagram of the hardware structure of the terminal/server according to the present invention.

With the foregoing drawings in mind, certain embodiments of the disclosure have been shown and described in more detail below. The drawings and written description are not intended to limit the scope of the disclosed concepts in any way, but rather to illustrate the disclosed concepts to those skilled in the art by reference to specific embodiments.

Detailed Description

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

The invention provides a remote control method which can be realized by a system architecture diagram shown in figure 1. As shown in fig. 1, the first terminal 100 is communicatively connected to the second terminal 200, and both the first terminal 100 and the second terminal 200 are communicatively connected to the server 300. The first terminal 100 and the second terminal 200 may be home appliances or control terminals, and both terminals have screens so that the first terminal 100 can perform screen sharing with the second terminal 200. The first terminal 100 and the second terminal 200 are screen sharing implemented by the server 300, that is, the second terminal 200 transmits contents displayed on the screen to the server 300, and the server 300 forwards the contents to the first terminal 100. The first terminal 100 and the second terminal 200 may perform real-time sharing of a screen, where the real-time sharing is that the server 300 transmits content of the screen of the second terminal 200 to the first terminal 100 in real time, and the first terminal 100 and the second terminal 200 may perform a voice call through the server, the server 300 feeds back coordinates of an operation performed by the first terminal 100 to the second terminal 200, and the second terminal 200 performs the same operation as that performed by the user of the first terminal 100 according to the coordinates. The first terminal 100 and the second terminal 200 may also perform delayed screen sharing, the second terminal 200 may record a screenshot of an operation performed by a user on a screen of the second terminal 200, and upload the screenshot to the server 300, the server 300 generates a media stream from all the screenshots, the server 300 notifies the first terminal 100 of the generation of the media stream, if the user of the first terminal 100 is idle, the media stream may be downloaded from the server 300, and identifies a device and a function of the device that are required to be controlled by the second terminal 200 through the played media stream, so as to perform a corresponding operation on the first terminal 100, thereby generating a corresponding control instruction, the first terminal 100 sends the control instruction to the server 300 without sending the operation performed by the screen of the first terminal 100, and the server 300 sends the control instruction to the device that is required to be controlled, thereby implementing the delayed screen sharing between the first terminal 100 and the second terminal 200.

The following describes the technical solutions of the present invention and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.

Referring to fig. 2, fig. 2 is a diagram of a remote control method according to a first embodiment of the present invention, applied to a first terminal, the remote control method including the steps of:

and step S10, after the first terminal shares the control function of the second terminal on the equipment, receiving a media stream transmitted by the server, wherein the media stream is generated by the server according to a screenshot transmitted by the second terminal, and the screenshot is used for recording the operation of a user on a control page on the second terminal.

In this embodiment, the execution subject is the first terminal. The first terminal may be a device having a screen, such as a mobile phone, a computer, or a home appliance. The first terminal may take over the second terminal, where taking over means that the first terminal shares a control function of the second terminal on the device, that is, the device may respond to a control instruction sent by the first terminal and may also respond to a control instruction sent by the second terminal. The equipment refers to household appliances such as washing machines and televisions. The second terminal is the party requesting help and the first terminal is the party helping.

The second terminal has a take-over function, and after the take-over function is started, the first terminal can share the control function of the second terminal on the equipment. After the takeover function is started, the second terminal can start screen sharing, and the screen sharing comprises two sharing modes. A sharing mode is real-time sharing, namely a second terminal transmits the content of a screen of the second terminal to a first terminal through a server, and the first terminal and the second terminal can carry out real-time voice, so that the first terminal can conveniently carry out remote control on the second terminal. The other sharing mode is delayed sharing, wherein delayed sharing means that the operation of a user on a control page of the second terminal is captured by the second terminal, all the captured images are sent to the server, and the user can leave without waiting for the feedback of the first terminal after the operation. The present embodiment describes delay sharing in detail.

When the second terminal starts the delay sharing, the second terminal performs screenshot when detecting the operation of the user on the control page, that is, the screenshot is used for recording the operation of the user on the control page on the second terminal. The control page is an interface displayed on the screen of the second terminal, and the user can control the equipment and the functions of the equipment through the control page. And the second terminal sends the screenshots to the server one by one, the server generates a media stream according to the sending time point of the screenshots after receiving all the screenshots, and the media stream records the action of the user on the control page.

And the server sends the generated media stream to the first terminal. The real-time sharing of the second terminal is to share the screen of the second terminal to the second terminal in real time, so that the second terminal can be determined to start the delayed sharing when the first terminal receives the media stream.

Step S20, playing a media stream on the shared page, where the media stream includes a target device to be controlled by the second terminal and a target function to be controlled by the target device.

The first terminal is provided with a shared page, the shared page is an H5 page, and the shared page is generated based on the control page of the second terminal, that is, the operation of the user of the first terminal on the shared page is equivalent to the operation of the user of the second terminal on the control page. And the first terminal plays the media stream on the shared page. Since the media stream is generated by each screenshot, there is an action of the second terminal operating on the control page in the media stream, and the action can determine the target device to be controlled and the target function to be controlled by the target device at the second terminal by the user, that is, the media stream includes the target device to be controlled and the target function to be controlled of the target device at the second terminal.

The user of the first terminal can determine the target equipment required to be controlled by the user of the second terminal and the target function to be controlled by the target equipment based on the played media stream. The user of the first terminal may perform an operation on the shared page, that is, perform a control operation of the target function on the shared page, and the first terminal may generate a control instruction of the target function based on the first control operation.

And step S30, when the control operation aiming at the target function on the shared page is detected, generating a control instruction of the target function, and transmitting the control instruction to the target device through the server.

If the first terminal detects the control operation aiming at the target function on the shared page, the control instruction of the target function can be generated, and then the control instruction is transmitted to the target equipment through the server, so that the target equipment responds to the control instruction, the first terminal helps the second terminal to control the target function of the target equipment, and the help seeking of the user of the first terminal is completed. It should be noted that, because the sharing mode of the first terminal is delay sharing, the user of the first terminal may not wait online, and therefore, the screen of the operation of the shared page on the second terminal does not need to be sent to the first terminal through the server.

In the technical scheme provided in the embodiment of the present year, after a first terminal shares a control function of a second terminal on a device, a media stream generated by a server based on all screenshots, sent by the second terminal, recorded for a user to operate on a control page is received, the media stream is played on the shared page so that the user corresponding to the first terminal can identify a target device to be controlled by the user of the second terminal and a control operation of the target device, a control instruction corresponding to the target function is generated when the control operation of the target function on the target device on the shared page is detected, and the control instruction is sent to the target device through the server. In the invention, the server caches all screenshots transmitted by the second terminal to generate the media stream, so that the first terminal can receive the media stream without image distortion and pause, and can identify the functions of the equipment which is required to be controlled by a user and corresponds to the second terminal through the media stream, so that the second terminal can accurately and timely control the equipment which is required to be controlled by the second terminal, and the reliability of remote control of household appliances is improved.

In one embodiment, step S30 includes:

generating icon information corresponding to the control instruction;

and transmitting the icon information to a second terminal through the server, and transmitting the control instruction to the target equipment through the server, wherein the second terminal adds the icon corresponding to the icon information to the control page, and when the touch operation of the icon is detected, the second terminal sends the control instruction to the target equipment.

In this embodiment, after the first terminal generates the control instruction, the first terminal generates icon information of the control instruction, and sends the icon information to the second terminal through the server. The icon information is data encapsulated by the icon corresponding to the control instruction. The control command may be multiple, that is, the control of the target function may require multiple control commands to achieve the control of the target function. And the first terminal forms each control instruction into an icon, the icon is a key icon, and the icon is endowed with a function of controlling the target function. The first terminal sends the icon to the second terminal. And the second terminal outputs prompt information whether the icon is stored or not, if the second terminal agrees to store the icon, the second terminal adds the icon to the control page, and the user can click the icon through the control page, so that the target function is directly controlled without performing multiple operations to obtain a plurality of control instructions to realize the control of the target function. The user can rename the icon on the control page, and further know the role of the icon through the renamed name. It can be understood that the second terminal adds the icon corresponding to the icon information to the control page, and when the touch operation of the icon is detected, the second terminal sends a control instruction to the target device. And analyzing the icon information to obtain an icon corresponding to the icon information.

In the technical scheme provided by this embodiment, the first terminal generates the icon corresponding to the control instruction, and transmits the icon to the second terminal through the server, so that the user of the second terminal can directly control the target function of the target device through the icon, and the user of the second terminal is prevented from repeatedly seeking help from the first terminal to control the same function.

In an embodiment, before step S10, the method further includes:

receiving a takeover request sent by a second terminal;

acquiring an encrypted message according to the take-over request, and carrying out take-over verification according to the encrypted message;

and when the taking-over check is passed, sending the prompt information that the taking-over check is passed to the second terminal, wherein the prompt information that the taking-over check is passed is used for the second terminal to authorize the first terminal to share the control function of the second terminal.

In this embodiment, in consideration of the security of the user data and the device, the first terminal starts the takeover function, and specific encryption verification needs to be performed between the first terminal and the second terminal. In addition, in order to facilitate the request initiator to use the takeover function, the takeover function of the first terminal needs to be made into a one-key takeover triggering key, after the first terminal and the second terminal pass the verification, the takeover function of the first terminal is successfully authorized, and the first terminal can start the takeover function.

Specifically, the second terminal acquires the check data, encrypts the check data to obtain an encrypted message, generates a take-over request based on the encrypted message, and sends the take-over request to the first terminal. After receiving the takeover request, the first terminal analyzes the takeover request to obtain an encrypted message, decrypts the encrypted message to obtain check data, then carries out takeover check by using the check data, if the check data is the same as the check data stored in the first terminal, the takeover check is passed, the first terminal sends prompt information that the takeover check is passed to the second terminal, and the second terminal receives the prompt information, authorizes the first terminal to share the control function of the second terminal on the equipment, namely, the second terminal successfully starts the takeover function.

In one embodiment, step S10 includes:

when receiving prompt information generated by a media stream sent by a server, stopping sending a picture displayed by a first terminal to a second terminal through the server, and sending a first acquisition request to the server;

In this embodiment, the user of the first terminal may delay help the second terminal to control the device. After the server produces the media stream, the server sends prompt information generated by the media stream to the first terminal, and prompts a user of the first terminal to help the second terminal control the equipment. It should be noted that, when the first terminal receives the prompt message generated by the media stream, the first terminal may determine that the second terminal is the delayed sharing, that is, the user of the second terminal is not online, so that the first terminal wirelessly shares the screen with the second terminal, and when the first terminal receives the prompt message generated by the media stream sent by the server, the first terminal stops sending the screen displayed by the first terminal to the second terminal through the server.

After the user of the first terminal is idle, the prompt message can be clicked, and the first terminal sends a first acquisition request to the server. The first acquisition request is used for acquiring the media stream from the server. And the server feeds back a response of the first acquisition request to the first terminal after receiving the first acquisition request, wherein the response of the first acquisition request comprises the media stream. That is, the first terminal receives a response of the first acquisition request sent by the server, and may acquire the media stream.

In one embodiment, step S10 includes:

sending a second acquisition request to the server;

and receiving a response of a second acquisition request sent by the server, wherein the response of the second acquisition request comprises the media stream.

In this embodiment, the user of the first terminal may help the second terminal control the device immediately, that is, the user of the first terminal actively requests the server to acquire the media stream.

Specifically, the user of the first terminal inputs a second acquisition request on the shared page, where the second acquisition request is used to request to acquire the media stream, that is, the first terminal acquires the second acquisition request. And the first terminal sends the second acquisition request to the server. After generating the media stream, the server sends a response of the second acquisition request to the first terminal, where the response of the second acquisition request includes the media stream.

The invention also provides a remote control method.

Referring to fig. 3, fig. 3 is a diagram illustrating a remote control method according to a second embodiment of the present invention, applied to a second terminal, the remote control method including:

and S40, when the second terminal authorizes the first terminal to share the control function of the second terminal on the equipment and detects the operation of the control page aiming at the second terminal, generating a screenshot corresponding to the control page, wherein the screenshot is used for recording the operation of the user on the control page.

In this embodiment, the execution subject is the second terminal. The second terminal may be a device having a screen, such as a mobile phone, a computer, or a home appliance. The first terminal may take over the second terminal, where taking over means that the first terminal shares a control function of the second terminal on the device, that is, the device may respond to a control instruction sent by the first terminal and may also respond to a control instruction sent by the second terminal. The equipment refers to household appliances such as washing machines and televisions. The second terminal is the party requesting help and the first terminal is the party assisting.

The second terminal has a take-over function, and after the take-over function is started, the first terminal can share the control function of the second terminal on the equipment. After the takeover function is started, the second terminal can start screen sharing, and the screen sharing comprises two sharing modes. A sharing mode is real-time sharing, namely a second terminal transmits the content of a screen of the second terminal to a first terminal through a server, and the first terminal and the second terminal can carry out real-time voice, so that the first terminal can conveniently carry out remote control on the second terminal. The other sharing mode is delayed sharing, the delayed sharing means that the operation of the user on the control page of the second terminal is captured by the second terminal, all the captured images are sent to the server, and the user can leave without waiting for the feedback of the first terminal after the operation. The present embodiment describes delay sharing in detail.

When the second terminal starts the delay sharing, that is, the second terminal authorizes the first terminal to share the control function of the second terminal on the device, and the second terminal generates a screenshot corresponding to the control page when detecting the operation on the control page, that is, the screenshot is performed on the operation of the user on the control page.

The control page is an interface displayed on the screen of the second terminal, and the user can control the equipment and the functions of the equipment through the control page.

And S50, transmitting each screenshot to a server, wherein each screenshot is used for the server to generate a media stream, and the first terminal controls the target function of the target equipment according to the media stream sent by the server.

And the second terminal sends the screenshots to the server one by one, the server generates a media stream according to the sending time point of the screenshots after receiving all the screenshots, and the media stream records the action of the user on the control page. It should be noted that the server may compress the screenshot, so as to generate a media stream with a small data volume.

The user of the first terminal can determine the target device which the second terminal user needs to control and the target function to be controlled by the target device based on the played media stream. The user of the first terminal may perform an operation on the shared page, that is, perform a control operation of the target function on the shared page, and the first terminal may generate a control instruction of the target function based on the first control operation.

And the control instruction of the function is transmitted to the target equipment through the server.

And step S60, receiving the prompt information of the target function control sent by the target equipment controlled by the second terminal, wherein the prompt information of the target function control is sent by the first terminal after the target function of the target equipment is controlled by the first terminal.

After the target function of the target device responds to the control instruction, the target device sends the prompt message controlled by the target function to the second terminal, that is, the prompt message controlled by the target function is sent by the first terminal after the target function of the target device is controlled by the first terminal. The prompt message is used for prompting the user of the first terminal to complete the control of the target device.

In the technical solution provided in this embodiment, when the second terminal authorizes the first terminal to share the control function of the second terminal on the device, and the user operates the control page of the second terminal, the screenshot corresponding to the control page is generated, and each screenshot is sent to the server, so that the server generates the media stream based on the screenshot, and thus the second terminal realizes control over the target function of the target device based on the media stream. In the invention, the server caches all screenshots transmitted by the second terminal to generate the media stream, so that the first terminal can receive the media stream without image distortion and pause, and can identify the functions of the equipment which is required to be controlled by a user and corresponds to the second terminal through the media stream, so that the second terminal can accurately and timely control the equipment which is required to be controlled by the second terminal, and the reliability of remote control of household appliances is improved.

In an embodiment, after step S60, the method further includes:

receiving icon information sent by a server;

and adding an icon corresponding to the icon information on the control page, wherein the second terminal sends a control instruction to the target equipment when detecting the touch operation of the icon.

In this embodiment, after the first terminal generates the control instruction, the first terminal generates icon information of the control instruction, and sends the icon information to the second terminal through the server. The icon information is data encapsulated by the icon corresponding to the control instruction. The control command may be multiple, that is, the control of the target function may require multiple control commands to achieve the control of the target function. And the first terminal forms each control instruction into an icon, the icon is a key icon, and the icon is endowed with a function of controlling the target function. And the first terminal sends the icon to the second terminal. And the second terminal outputs prompt information of whether the icon is stored or not, if the second terminal agrees to store the icon, the second terminal adds the icon to the control page, and the user can click the icon through the control page, so that the target function is directly controlled without performing multiple operations to obtain multiple control instructions to realize the control of the target function. The user can rename the icon on the control page, and further know the role of the icon through the renamed name. It can be understood that the second terminal adds the icon corresponding to the icon information to the control page, and when the touch operation of the icon is detected, the second terminal sends a control instruction to the target device. And analyzing the icon information to obtain an icon corresponding to the icon information.

In the technical scheme provided by this embodiment, the second terminal receives the icon corresponding to the control instruction generated by the first terminal, and adds the icon to the control page, so that the user of the second terminal can directly control the target function of the target device through the icon, and the user of the second terminal is prevented from repeatedly seeking help from the first terminal to control the same function.

In an embodiment, before step S40, the method further includes:

sending a take-over request to the first terminal, wherein the encrypted message is used for the first terminal to carry out take-over verification;

and when receiving prompt information which is sent by the second terminal and passes the verification, sending authorization information to the second terminal, wherein the authorization information is used for the second terminal to authorize the first terminal to share the control function of the second terminal on the equipment.

Specifically, the second terminal acquires the check data, encrypts the check data to obtain an encrypted message, generates a take-over request based on the encrypted message, and sends the take-over request to the first terminal. After receiving the takeover request, the first terminal analyzes the takeover request to obtain an encrypted message, decrypts the encrypted message to obtain check data, and then adopts the check data to carry out takeover check, if the check data is the same as the check data stored in the first terminal, the takeover check is passed, the first terminal sends prompt information that the takeover check is passed to the second terminal, and the second terminal receives the prompt information, sends authorization information to the second terminal, namely the authorization information is used for authorizing the first terminal to share the control function of the second terminal on the equipment, namely the second terminal successfully starts the takeover function.

In one embodiment, step S50 includes:

outputting a selection interface, wherein the selection interface displays the sharing mode of the second terminal sharing control function;

acquiring selection information of a selection interface;

and sending the sharing information to a server, wherein the sharing information is used for indicating that the sharing mode is delayed sharing.

In this embodiment, after the second terminal authorizes the first terminal to share the control function of the second terminal on the device, the second terminal may output a selection interface, and the selection interface displays a sharing mode of the second terminal sharing the control function, that is, the user may select real-time sharing or delayed sharing in the selection interface.

After the second terminal outputs the selection interface, the terminal of the second user can perform selection operation on the selection interface, so that the selection interface generates selection information corresponding to the selection operation. The selection information is the sharing mode. And when the selection information is the delay sharing, namely when the sharing mode is determined to be the delay sharing based on the selection information, generating the sharing information corresponding to the delay sharing. And the second terminal sends the sharing information to the server, so that the server determines that the sharing mode of the second terminal is delayed sharing based on the sharing information, namely, the server compresses the screenshot and generates a media stream after receiving all the screenshots. If the server does not receive the shared information, the server directly forwards the picture transmitted by the second terminal to the first terminal in real time without generating a media stream.

Referring to fig. 4, fig. 4 is a third embodiment of a remote control method of the present invention, applied to a server, the remote control method including:

and step S70, after the first terminal shares the control function of the second terminal on the equipment, receiving all screenshots transmitted by the second terminal, wherein the screenshots are used for recording the operation of the user on the control page on the second terminal.

In this embodiment, the execution subject is a server. The first terminal can share the control function of the second terminal on the equipment through the server.

The second terminal and the second terminal may be devices having screens such as mobile phones, computers, home appliances, and the like. The first terminal may take over the second terminal, where taking over means that the first terminal shares a control function of the second terminal on the device, that is, the device may respond to a control instruction sent by the first terminal and may also respond to a control instruction sent by the second terminal. The equipment refers to household appliances such as washing machines and televisions. The second terminal is the party requesting help and the first terminal is the party helping.

When the second terminal starts the delay sharing, that is, the second terminal authorizes the first terminal to share the control function of the second terminal on the device, and the second terminal generates a screenshot corresponding to the control page when detecting the operation of the user on the control page, that is, the screenshot is performed on the operation of the user on the control page.

And the second terminal sends the screenshots to the server one by one, namely the server receives the screenshots transmitted by the second terminal.

And S80, generating a media stream according to each screenshot, and sending the media stream to the first terminal, wherein the media stream comprises target equipment to be controlled by the second terminal and target functions to be controlled by the target equipment.

After receiving all the screenshots, the server generates a media stream according to the sending time point of the screenshots, wherein the media stream records the action of the user on the control page, namely the media stream comprises target equipment to be controlled by the second terminal and target functions to be controlled by the target equipment. It should be noted that the server may compress the screenshot, so as to generate a media stream with a small data volume.

And step S90, receiving a control instruction sent by the first terminal, and sending the control instruction to the target equipment, wherein the control instruction is generated when the second terminal detects a control operation aiming at a target function on the shared page.

The first terminal is provided with a shared page, the shared page is an H5 page, and the shared page is generated based on the control page of the second terminal, that is, the operation of the user of the first terminal on the shared page is equivalent to the operation of the user of the second terminal on the control page. And the first terminal plays the media stream on the shared page. The media stream is generated by each screenshot, so that the action of the second terminal on operating the control page is in the media stream, and the action can determine the target equipment to be controlled by the user on the second terminal and the target function to be controlled by the target equipment, namely the media stream comprises the target equipment to be controlled by the second terminal and the target function to be controlled by the target equipment.

If the first terminal detects the control operation aiming at the target function on the shared page, the control instruction of the target function can be generated, and then the control instruction is transmitted to the target equipment through the server, so that the target equipment responds to the control instruction, the first terminal helps the second terminal to control the target function of the target equipment, and the help seeking of the user of the first terminal is completed. It should be noted that, since the sharing mode of the first terminal is delay sharing, the user of the first terminal may not wait online, and therefore, the screen of the operation of the shared page on the second terminal does not need to be sent to the first terminal through the server.

In the technical scheme provided by this embodiment, after the first terminal shares the control function of the second terminal on the device, the server receives each screenshot transmitted by the second terminal and used for recording the operation of the user on the control page on the second terminal, and then generates a media stream according to each screenshot, and sends the media stream to the first terminal, so that the first terminal generates a control instruction for controlling the function of the target device based on the media stream, and then the server forwards the control instruction to the target device. In this embodiment, the server caches all screenshots transmitted by the second terminal to generate a media stream, so that the first terminal does not have image distortion and unsmooth blocking when receiving the media stream, and the function of the device to be controlled by the user corresponding to the second terminal can be identified through the media stream, so that the second terminal can accurately and timely control the device to be controlled by the second terminal, and the reliability of remote control of the household appliance device is improved.

In one embodiment, step S90 includes:

receiving a control instruction sent by a first terminal and icon information corresponding to the control instruction;

and sending the control instruction to the target equipment, and sending the icon information to the second terminal, wherein the second terminal adds the icon corresponding to the icon information to the control page, and sends the control instruction to the target equipment when the touch operation of the icon is detected.

In this embodiment, after the first terminal generates the control instruction, icon information of the control instruction is generated, and the icon information is sent to the second terminal through the server. The icon information is data encapsulated by the icon corresponding to the control instruction. The control command may be multiple, that is, the control of the target function may require multiple control commands to achieve the control of the target function. And the first terminal forms each control instruction into an icon, the icon is a key icon, and the icon is endowed with a function of controlling the target function. The first terminal sends the icon to the second terminal. And the second terminal outputs prompt information whether the icon is stored or not, if the second terminal agrees to store the icon, the second terminal adds the icon to the control page, and the user can click the icon through the control page, so that the target function is directly controlled without performing multiple operations to obtain a plurality of control instructions to realize the control of the target function. The user can rename the icon on the control page, and further know the role of the icon through the renamed name. It can be understood that the second terminal adds the icon corresponding to the icon information to the control page, and when the touch operation of the icon is detected, the second terminal sends a control instruction to the target device. And analyzing the icon information to obtain an icon corresponding to the icon information.

In the technical scheme provided by this embodiment, the server receives the icon corresponding to the control instruction generated by the first terminal, and transmits the icon to the second terminal, so that the user of the second terminal can directly control the target function of the target device through the icon, and the user of the second terminal is prevented from repeatedly seeking help from the first terminal to control the same function.

In one embodiment, step S80 includes:

receiving a first acquisition request sent by a first terminal;

In this embodiment, the user of the first terminal may delay to help the second terminal control the device. After the server produces the media stream, the server sends prompt information generated by the media stream to the first terminal, and prompts a user of the first terminal that the second terminal can control the equipment.

After the user of the first terminal is idle, the prompt message can be clicked, and the first terminal sends a first acquisition request to the server. The first acquisition request is used for acquiring the media stream from the server. And the server feeds back a response of the first acquisition request to the first terminal after receiving the first acquisition request, wherein the response of the first acquisition request comprises the media stream. That is, the first terminal receives a response of the first acquisition request sent by the server, and the first terminal can acquire the media stream.

In one embodiment, step S80 includes:

receiving a second acquisition request sent by the first terminal;

In one embodiment, step S70 includes:

receiving shared information sent by a second terminal;

and when determining that the first terminal delays to share the control function of the second terminal according to the shared information, generating a media stream according to each screenshot.

In this embodiment, after the second terminal authorizes the first terminal to share the control function of the second terminal to the device, the second terminal may output a selection interface, where the selection interface includes a sharing mode of the second terminal sharing the control function, that is, the user may select real-time sharing or delayed sharing in the selection interface.

After the second terminal outputs the selection interface, the terminal of the second user can perform selection operation on the selection interface, so that the selection interface generates selection information corresponding to the selection operation. The selection information is the sharing mode. And when the selection information is the delay sharing, namely when the sharing mode is determined to be the delay sharing based on the selection information, generating the sharing information corresponding to the delay sharing. And the second terminal sends the shared information to the server. The server analyzes the shared information after receiving the shared information, if the shared mode obtained by the analyzed shared information is delay sharing, that is, the server determines that the shared mode of the second terminal is delay sharing based on the shared information, the server does not forward the image sent by the second terminal to the first terminal in real time, but compresses the screenshot and generates a media stream after receiving all the screenshots. If the server does not receive the shared information, the server directly forwards the picture transmitted by the second terminal to the first terminal in real time without generating a media stream.

The present invention also provides a terminal, and referring to fig. 5, the terminal 500 includes:

a first receiving module 510, configured to receive a media stream transmitted by a server after a first terminal shares a control function of a second terminal on a device, where the media stream is generated by the server according to a screenshot transmitted by the second terminal, and the screenshot is used to record an operation of a user on a control page on the second terminal;

a playing module 520, configured to play a media stream on the shared page, where the media stream includes a target device to be controlled by the second terminal and a target function to be controlled by the target device;

the first generating module 530 is configured to, when a control operation for a target function on a shared page is detected, generate a control instruction for the target function, and transmit the control instruction to a target device through a server.

In one embodiment, the terminal 500 includes:

a first generating module 530, configured to generate icon information corresponding to the control instruction;

and the transmission module is used for transmitting the icon information to the second terminal through the server and transmitting the control instruction to the target equipment through the server, wherein the second terminal adds the icon corresponding to the icon information to the control page, and when the touch operation of the icon is detected, the second terminal sends the control instruction to the target equipment.

In one embodiment, the terminal 500 includes:

a first receiving module 510, configured to receive a takeover request sent by a second terminal;

the acquisition module is used for acquiring the encrypted message according to the take-over request and carrying out take-over check according to the encrypted message;

and the sending module is used for sending the prompt information that the verification of the take-over is passed to the second terminal when the take-over verification is passed, and the prompt information that the verification of the take-over is passed is used for the second terminal to authorize the first terminal to share the control function of the second terminal.

In one embodiment, the terminal 500 includes:

the sending module is used for stopping sending the picture displayed by the first terminal to the second terminal through the server when receiving the prompt message generated by the media stream sent by the server, and sending a first acquisition request to the server;

the first receiving module 510 is configured to receive a response of the first obtaining request sent by the server, where the response of the second obtaining request includes the media stream.

In one embodiment, terminal 500 includes:

the acquisition module is used for acquiring a second acquisition request input through the shared page, and the second acquisition request is used for requesting to acquire the media stream;

the sending module is used for sending a second acquisition request to the server;

the first receiving module 510 is configured to receive a response of the second obtaining request sent by the server, where the response of the second obtaining request includes the media stream.

The present invention also provides a terminal, and referring to fig. 6, the terminal 600 includes:

a second receiving module 610, configured to generate a screenshot corresponding to a control page when the second terminal authorizes the first terminal to share a control function of the second terminal for the device, and detects an operation on the control page of the second terminal, where the screenshot is used to record an operation of a user on the control page;

a transmission module 620, configured to transmit each screenshot to a server, where each screenshot is used for the server to generate a media stream, and the first terminal controls a target function of the target device according to the media stream sent by the server;

the second receiving module 610 is configured to receive a prompt message for target function control sent by a target device controlled by a second terminal, where the prompt message for target function control is sent by the first terminal after controlling a target function of the target device.

In one embodiment, the terminal 600 includes:

a second receiving module 610, configured to receive icon information sent by a server;

and the adding module is used for adding an icon corresponding to the icon information on the control page, wherein the second terminal sends a control instruction to the target device when detecting the touch operation of the icon.

In one embodiment, the terminal 600 includes:

the acquisition module is used for acquiring the encrypted message and generating a take-over request according to the encrypted message;

the sending module is used for sending the take-over request to the first terminal, wherein the encrypted message is used for the first terminal to carry out take-over verification;

and the sending module is used for sending authorization information to the second terminal when receiving the prompt message sent by the second terminal and used for authorizing the first terminal to share the control function of the second terminal on the equipment.

In one embodiment, the terminal 600 includes:

the output module is used for outputting a selection interface, wherein the selection interface displays the sharing mode of the second terminal for sharing the control function;

the acquisition module is used for acquiring selection information of the selection interface;

the generating module is used for generating the sharing information corresponding to the delayed sharing when the sharing mode is determined to be the delayed sharing according to the selection information;

and the sending module is used for sending the sharing information to the server, and the sharing information is used for indicating that the sharing mode is delayed sharing.

The present invention also provides a server, and referring to fig. 7, a server 700 includes:

a third receiving module 710, configured to receive, after the first terminal shares a control function of the second terminal on the device, each screenshot transmitted by the second terminal, where the screenshot is used to record an operation of a user on a control page on the second terminal;

a second generating module 720, configured to generate a media stream according to each screenshot, and send the media stream to the first terminal, where the media stream includes a target device to be controlled by the second terminal and a target function to be controlled by the target device;

the third receiving module 710 is configured to receive a control instruction sent by the first terminal, and send the control instruction to the target device, where the control instruction is generated when the second terminal detects a control operation for a target function on the shared page.

In one embodiment, the server 700 includes:

a third receiving module 710, configured to receive a control instruction sent by the first terminal and icon information corresponding to the control instruction;

and the sending module is used for sending the control instruction to the target equipment and sending the icon information to the second terminal, wherein the second terminal adds the icon corresponding to the icon information to the control page and sends the control instruction to the target equipment when the touch operation of the icon is detected.

In one embodiment, server 700 includes:

the sending module is used for sending the generated prompt information of the media stream to the first terminal;

a third receiving module 710, configured to receive a first obtaining request sent by a first terminal;

and the sending module is used for sending a response of the first acquisition request to the first terminal, wherein the response of the first acquisition request comprises the media stream.

In one embodiment, server 700 includes:

a third receiving module 710, configured to receive a second obtaining request sent by the first terminal;

and the sending module is used for sending a response of the second acquisition request to the first terminal, wherein the response of the second acquisition request comprises the media stream.

In one embodiment, the server 700 includes:

a third receiving module 710, configured to receive shared information sent by a second terminal;

and a second generating module 720, configured to generate a media stream according to each screenshot when it is determined that the first terminal delays to share the control function of the second terminal according to the shared information.

Fig. 8 is a diagram illustrating a hardware configuration of a terminal/server according to an exemplary embodiment.

The terminal/server 800 may include: a processor 81, such as a CPU, a memory 82, and a transceiver 83. Those skilled in the art will appreciate that the architecture shown in fig. 8 does not constitute a limitation of the terminal/server and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components. The memory 82 may be implemented by any type or combination of volatile or non-volatile storage devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disks.

The processor 81 may call computer programs stored in the memory 82 to perform all or part of the steps of the remote control method described above.

The transceiver 83 is used for receiving information transmitted from and transmitting information to an external device.

A non-transitory computer readable storage medium, in which control instructions, when executed by a processor of a terminal/server, enable the terminal/server to perform the above-described remote control method.

A computer program product comprising a computer program which, when executed by a processor of a terminal/server, enables the terminal/server to perform the remote control method described above.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It will be understood that the present disclosure is not limited to the precise arrangements that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A remote control method is applied to a first terminal, and is characterized by comprising the following steps:

when the control operation aiming at the target function on the shared page is detected, a control instruction of the target function is generated and transmitted to the target equipment through the server.

2. The remote control method according to claim 1, wherein the step of transmitting the control instruction to the target device through the server comprises:

generating icon information corresponding to the control instruction;

3. The remote control method according to claim 1, wherein before the step of receiving the media stream transmitted by the server after the first terminal shares the control function of the second terminal on the device, the method further comprises:

receiving a takeover request sent by the second terminal;

and when the take-over check is passed, sending a prompt message that the take-over check is passed to the second terminal, wherein the prompt message that the take-over check is passed is used for the second terminal to authorize the first terminal to share the control function of the second terminal.

4. A remote control method according to any one of claims 1 to 3, wherein the step of receiving the media stream transmitted by the server comprises:

5. The remote control method according to any one of claims 1 to 3, wherein the step of receiving the image information transmitted from the server comprises:

sending the second acquisition request to the server;

6. A remote control method is applied to a second terminal and is characterized by comprising the following steps:

and receiving prompt information of target function control sent by target equipment controlled by the second terminal, wherein the prompt information of target function control is sent by the first terminal after the target function of the target equipment is controlled by the first terminal.

7. The remote control method according to claim 6, wherein after the step of receiving the prompt message for target function control sent by the target device controlled by the second terminal, the method further comprises:

receiving icon information sent by the server;

8. The remote control method according to claim 6, wherein before the step in which the second terminal authorizes the first terminal to share the control function of the second terminal to the device, and when a user operates a control page of the second terminal, the step of generating a screenshot corresponding to the control page further comprises:

9. The remote control method according to any one of claims 6 to 8, wherein the step of transmitting each of the screenshots to the server comprises:

acquiring selection information of the selection interface;

10. A remote control method is applied to a server and is characterized by comprising the following steps:

after a first terminal shares a control function of a second terminal on equipment, receiving all screenshots transmitted by the second terminal, wherein the screenshots are used for recording the operation of a user on a control page on the second terminal;

11. The remote control method according to claim 10, wherein the step of receiving the control command sent by the first terminal and sending the control command to the target device comprises:

12. The remote control method according to claim 10, wherein the step of transmitting the media stream to the first terminal comprises:

sending a prompt message generated by the media stream to the first terminal;

receiving a first acquisition request sent by the first terminal;

13. The remote control method according to claim 10, wherein the step of transmitting the media stream to the first terminal comprises:

receiving a second acquisition request sent by the first terminal;

14. The remote control method according to any one of claims 10 to 13, wherein the step of generating a media stream from each of the screenshots comprises:

receiving shared information sent by the second terminal;

and when the first terminal is determined to share the control function of the second terminal in a delayed manner according to the sharing information, generating a media stream according to each screenshot.

15. A terminal, characterized in that the terminal comprises:

the first generation module is used for generating a control instruction of the target function when the control operation aiming at the target function on the shared page is detected, and transmitting the control instruction to the target device through the server.

16. A terminal, characterized in that the terminal comprises:

17. A server, characterized in that the server comprises:

a second generation module, configured to generate a media stream according to each screenshot, and send the media stream to the first terminal, where the media stream includes a target device to be controlled by the second terminal and a target function to be controlled by the target device;