WO2015117367A1

WO2015117367A1 - Remote assistance control method and device

Info

Publication number: WO2015117367A1
Application number: PCT/CN2014/088871
Authority: WO
Inventors: 周轶
Original assignee: 中兴通讯股份有限公司
Priority date: 2014-08-21
Filing date: 2014-10-17
Publication date: 2015-08-13
Also published as: CN105357240A

Abstract

Provided are a remote assistance control method and device. The method comprises: establishing, by a controlling mobile terminal, a connection to a controlled mobile terminal; receiving screen display information sent by the controlled mobile terminal and carrying an assistance content, and displaying the screen display information on a screen of the controlling mobile terminal; and according to the assistance content, sending an operation instruction which operates the screen display information displayed on the screen of the controlling mobile terminal to the controlled mobile terminal, and conducting remote assistance on the controlled mobile terminal using the operation instruction. By means of the present invention, the problem existing in the related art that remote assistance cannot be conducted between mobile terminals is solved, thereby achieving the effects of realizing remote assistance between mobile terminals and improving the user experience.

Description

Remote assistance control method and device

Technical field

The present invention relates to the field of communications, and in particular to a remote assistance control method and apparatus.

Background technique

Currently, the number of Android phones is growing at an alarming rate. Especially in the domestic mobile phone market, Android mobile phones are the most mobile phones in the market, and almost all mobile phone users have one. The reason why Android phones are so popular is that Android phones are a new generation of smart phones with a very powerful Android operating system. The source code for the operating system is fully developed and is available to all mobile phone manufacturers for free. At the same time, the software development platform of the operating system is very good, it is convenient for programmers to develop their own software on it, which has spawned a large number of programmers who develop software on the Android system, which makes the operating system have a huge software library.

For young mobile phone users, using an Android phone is like using a computer. You can freely configure your favorite desktops, apps, tools, games, movies, and more. But for older users, there are big problems. Due to the too complicated operation of the Android mobile phone, this part of the mobile phone group is discouraged from the Android mobile phone, so that the powerful function of the mobile phone cannot be truly experienced. For this part of the population, they need to use their mobile phones when they are in doubt or difficulty. Someone can help them operate their mobile phones, and then gradually become familiar with the various operations of the mobile phone, and experience the convenience and power of the Android mobile phone. However, in the prior art, remote assistance cannot be implemented between mobile terminals, and the problem will limit the promotion and development of Android mobile phones to a certain extent.

In view of the problem that remote assistance cannot be performed between mobile terminals in the prior art, an effective solution has not been proposed yet.

Summary of the invention

The present invention provides a remote assistance control method and apparatus to at least solve the problem that remote assistance cannot be performed between mobile terminals in the related art.

According to an aspect of the present invention, a remote assistance control method is provided, including: controlling a mobile terminal to establish a connection with a controlled mobile terminal; receiving, by the controlled mobile terminal, on-screen information carrying the assisted content, and Displaying the on-screen display information on the screen of the control mobile terminal; according to the assisted content, An operation instruction for displaying the on-screen display information on the screen of the control mobile terminal is sent to the controlled mobile terminal, and the controlled mobile terminal is remotely assisted by using the operation instruction.

Preferably, the controlling the mobile terminal to establish a connection with the controlled mobile terminal comprises: establishing, by the server, a channel for forwarding data between the controlled mobile terminal, comprising: controlling, by using the mobile terminal, The peer P2P connection and/or server message forwarding between the controlled mobile terminals establishes at least one of the following channels: a protocol command data channel, a real-time screen image differential data channel, and a voice data channel.

Preferably, before the controlling the mobile terminal establishes the connection with the controlled mobile terminal, the method further includes: receiving a call request of the controlled mobile terminal sent by the server; determining whether the controlled mobile terminal that initiates the call is The controlling the terminal in the remote assistance list of the mobile terminal; if the determination result is yes, initiating a request for remotely assisting the controlled mobile terminal to the server.

Preferably, receiving the screen display information that is sent by the controlled mobile terminal and carrying the assisted content, and displaying the screen display information on the screen of the controlling mobile terminal, comprises: receiving, sending, by the controlled mobile terminal The screen display information carrying the assisted content and subjected to the differential processing; parsing the received screen display information, and displaying the parsed screen display information on the screen of the control mobile terminal on.

Preferably, the operation instruction comprises at least one of the following: a voice operation instruction, a gesture touch operation instruction, and a key operation instruction.

According to another aspect of the present invention, a remote assistance control method is provided, including: a controlled mobile terminal establishes a connection with a control mobile terminal; and transmits on-screen information carrying the assisted content to the control mobile terminal; receiving And controlling, by the mobile terminal, an operation instruction for operating the screen display information of the controlled mobile terminal according to the assistance content; and operating the controlled mobile terminal according to the operation instruction.

Preferably, the establishing, by the controlled mobile terminal, the connection with the controlling the mobile terminal comprises: establishing, by the server, a channel for forwarding data between the controlling mobile terminal, comprising: passing the controlled mobile terminal and the The control of the peer P2P connection and/or the server message forwarding between the mobile terminals establishes at least one of the following channels: a protocol command data channel, a real-time screen image differential data channel, and a voice data channel.

Preferably, the transmitting the display information carrying the assisted content to the controlling mobile terminal comprises: performing differential processing on the screen display information; and sending the screen display information after the differential processing to the Control the mobile terminal.

According to still another aspect of the present invention, a remote assistance control apparatus is provided for controlling a mobile terminal side, comprising: a first establishing module configured to establish a connection with the controlled mobile terminal; and a display module configured to receive the receiving Displaying the screen display information carried by the controlled mobile terminal and carrying the assisted content, and displaying the on-screen display information on the screen of the control mobile terminal; the assisting module is configured to display the display in accordance with the assisted content An operation instruction for controlling the on-screen information on the screen of the mobile terminal to be operated is sent to the controlled mobile terminal, and the controlled mobile terminal is remotely assisted by using the operation instruction.

Preferably, the first establishing module includes: a first establishing unit, configured to establish, by using a server, a channel for forwarding data with the controlled mobile terminal, including: controlling, by using the mobile terminal, the The peer P2P connection and/or server message forwarding between the mobile terminals is established to establish at least one of the following channels: a protocol command data channel, a real-time screen image differential data channel, and a voice data channel.

Preferably, the remote assistance control device further includes: a first receiving module, configured to receive a call request of the controlled mobile terminal sent by the server; and a determining module, configured to determine whether the controlled mobile terminal that initiates the call is And controlling, by the mobile terminal, a terminal in the remote assistance list; and the initiating module is configured to initiate a request for remotely assisting the controlled mobile terminal to the server if the determination result of the determining module is yes.

Preferably, the display module includes: a receiving unit configured to receive the screen display information that is sent by the controlled mobile terminal and that carries the assisted content and that has undergone differential processing; and the display unit is configured to receive the received The screen display information is subjected to parsing processing, and the parsed screen display information is displayed on the screen of the control mobile terminal.

According to still another aspect of the present invention, a remote assistance control apparatus is provided, which is applied to a controlled mobile terminal side, and includes: a second establishing module configured to establish a connection with the controlling mobile terminal; and a sending module configured to be carried The screen display information with the assisted content is sent to the control mobile terminal; the second receiving module is configured to receive the operation of the control mobile terminal to perform operation on the screen display information of the controlled mobile terminal according to the assisted content An operation module configured to operate the controlled mobile terminal according to the operation instruction.

Preferably, the second establishing module includes: a second establishing unit, configured to establish, by the server, a channel for forwarding data with the controlling mobile terminal, including: by the controlled mobile terminal and the controlling The peer P2P connection and/or server message forwarding between the mobile terminals establishes at least one of the following channels: a protocol command data channel, a real-time screen image differential data channel, and a voice data channel.

Preferably, the sending module includes: a processing unit configured to perform differential processing on the screen display information; and a sending unit configured to send the screen display information after the differential processing to the control mobile terminal.

According to the present invention, the mobile terminal is controlled to establish a connection with the controlled mobile terminal; the screen display information carried by the controlled mobile terminal carrying the assisted content is received, and the screen display information is displayed on the control mobile terminal. On the screen, according to the assistance content, an operation instruction for operating the on-screen information displayed on the screen of the mobile control terminal is sent to the controlled mobile terminal, and the operation instruction is used to Controlling the mobile terminal for remote assistance solves the problem that the remote terminals cannot be remotely assisted in the related art, thereby achieving the effect of realizing remote assistance between the mobile terminals and improving the user experience.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:

1 is a flow chart of a first method of controlling remote assistance according to an embodiment of the present invention;

2 is a flowchart of a second method of controlling remote assistance according to an embodiment of the present invention;

3 is a structural block diagram of a remote assistance control device 1 according to an embodiment of the present invention;

4 is a structural block diagram of a first establishing module 32 in a remote assistance control device 1 according to an embodiment of the present invention;

FIG. 5 is a block diagram showing a preferred structure of a remote assistance control apparatus 1 according to an embodiment of the present invention; FIG.

6 is a structural block diagram of a display module 34 in a remote assistance control device 1 according to an embodiment of the present invention;

FIG. 7 is a structural block diagram of a remote assistance control apparatus 2 according to an embodiment of the present invention; FIG.

FIG. 8 is a structural block diagram of a second establishing module 72 in the remote assistance control apparatus 2 according to an embodiment of the present invention;

FIG. 9 is a structural block diagram of a transmitting module 74 in a remote assisted control device 2 according to an embodiment of the present invention;

10 is an interaction diagram of a first connection between a control end and a controlled end according to an embodiment of the present invention;

11 is an interaction diagram of control and host control assistance according to an embodiment of the present invention.

detailed description

The invention will be described in detail below with reference to the drawings in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict.

In the embodiment, a remote assistance control method is provided. FIG. 1 is a flowchart of a remote assistance control method according to an embodiment of the present invention. As shown in FIG. 1, the process includes the following steps:

Step S102, controlling the mobile terminal to establish a connection with the controlled mobile terminal;

Step S104, receiving the screen display information carried by the controlled mobile terminal and carrying the assisted content, and displaying the screen display information on the screen of the control mobile terminal;

Step S106: Send an operation instruction for operating the on-screen display information displayed on the screen of the control mobile terminal to the controlled mobile terminal according to the assistance content, and use the operation instruction to perform remote assistance on the controlled mobile terminal.

Through the above steps, the mobile terminal is controlled to establish a connection with the controlled mobile terminal; the screen display information carried by the controlled mobile terminal carrying the assisted content is received, and the screen display information is displayed on the screen of the control mobile terminal; Content, sending an operation instruction for displaying the on-screen information displayed on the screen of the control mobile terminal to the controlled mobile terminal, and using the operation instruction to remotely assist the controlled mobile terminal, thereby realizing remote assistance by the control mobile terminal Controlling the mobile terminal solves the problem that the remote terminal cannot be remotely assisted in the related art, thereby achieving the effect of realizing remote assistance between the mobile terminals and improving the user experience.

In a preferred embodiment, the mobile terminal is controlled to establish a channel for forwarding data between the controlled mobile terminal and the controlled mobile terminal by establishing, by the server, the mobile terminal and the controlled mobile terminal. The peer P2P connection and/or server message forwarding between the terminals establishes at least one of the following channels: a protocol command data channel, a real-time screen image differential data channel, and a voice data channel. That is, the connection between the control terminal and the mobile terminal is established by the server. After the two parties are connected, it is possible to start the control of the remote assistance between the mobile terminals.

In an optional embodiment, before controlling the mobile terminal to establish a remote assistance connection with the controlled mobile terminal, the controlling mobile terminal can learn that the controlled mobile terminal needs remote assistance by controlling the mobile terminal to receive the sending by the server. The call request of the controlled mobile terminal; determining whether the controlled mobile terminal that initiated the call is controlling the terminal in the remote assistance list of the mobile terminal; and if the determination result is yes, initiating a request for remotely assisting the controlled mobile terminal to the server. In this embodiment, a related control for remote assistance may be installed in the controlled mobile terminal, and the control is used to initiate a call request to the control terminal, and the mobile terminal is controlled to connect the call. After the call is terminated within a short time, the call can be terminated in one second. Of course, other methods can also be used to know that the controlled mobile terminal needs to perform remote assistance. For example, the received mobile terminal can learn that the controlled mobile terminal needs remote assistance, so that when the controlled mobile terminal needs remote assistance, it can be realized. Timely remote assistance for the controlled mobile terminal.

In the process of performing remote assistance control on the controlled mobile terminal, controlling the mobile terminal to receive the screen display information carried by the controlled mobile terminal and carrying the assisted content, and displaying the screen display information on the screen of the control mobile terminal includes: receiving The screen display information sent by the controlled mobile terminal carrying the assisted content and subjected to the differential processing; parsing the received screen display information, and displaying the parsed screen display information on the screen of the control mobile terminal. In this way, the mobile terminal can be controlled to know which icons are on the interface of the controlled mobile terminal, and where the icons are distributed, and to better know the changed portion of the screen of the controlled mobile terminal, and thus conveniently The controlled mobile terminal controls.

As can be seen from FIG. 1 , the mobile terminal is controlled by the operation instruction to remotely assist the controlled mobile terminal, and the operation instruction may include at least one of the following: a voice operation instruction, a gesture touch operation instruction, and a key operation instruction. Wherein, when the controlled mobile terminal is controlled by using a voice operation instruction, the voice recognition module may be first recognized by the voice recognition module, converted into behavior data that can be operated on the terminal, and then the behavior data is sent to the controlled To the mobile terminal, thereby implementing remote assistance to the controlled mobile terminal. For the gesture touch operation instruction and the key operation instruction, it can be converted into behavior data, and then sent to the controlled mobile terminal, and the controlled mobile terminal converts the behavior data into related operations on the controlled terminal, thereby implementing Controlling the purpose of remote assistance of the mobile terminal to the controlled mobile terminal.

FIG. 2 is a flowchart of a second method for controlling remote assistance according to an embodiment of the present invention. As shown in FIG. 2, the process includes the following steps:

Step S202, the controlled mobile terminal establishes a connection with the control mobile terminal;

Step S204, the screen display information carrying the assisted content is sent to the control mobile terminal;

Step S206, receiving an operation instruction for controlling, by the mobile terminal, to perform operation on the screen display information of the controlled mobile terminal according to the assisted content;

Step S208, operating the controlled mobile terminal according to the operation instruction.

Through the above steps, the controlled mobile terminal is used to establish and control the connection of the mobile terminal; the screen display information carrying the assisted content is sent to the control mobile terminal; and the screen display information sent by the controlled mobile terminal according to the assisted content to the controlled mobile terminal is received and controlled. An operation instruction for performing an operation; a party that operates the controlled mobile terminal according to the operation instruction The method realizes remotely assisting the controlled mobile terminal by controlling the mobile terminal, and solves the problem that the remote terminal cannot be remotely assisted in the related art, thereby achieving the effect of realizing remote assistance between the mobile terminals and improving the user experience.

In a preferred embodiment, establishing, by the controlled mobile terminal, the connection with the controlling mobile terminal may include the steps of: establishing and controlling, by the server, a channel for forwarding data between the mobile terminal, including: controlling the mobile terminal and controlling The peer P2P connection and/or server message forwarding between the mobile terminals establishes at least one of the following channels: a protocol command data channel, a real-time screen image differential data channel, and a voice data channel. That is, the connection between the control terminal and the mobile terminal is established by the server, and after the two parties are connected, the control of the remote assistance between the mobile terminals can be started.

In a preferred embodiment, when the screen display information carrying the assisted content is sent to the control mobile terminal, the screen display information may be differentially processed, and the screen display information after the differential processing is sent to the control. Mobile terminal. Therefore, the control mobile terminal can clearly know the changed portion of the screen of the controlled mobile terminal, thereby conveniently controlling the controlled mobile terminal.

The operation instruction received by the controlled mobile terminal may be at least one of the following instructions, a voice operation instruction, a gesture touch operation instruction, and a key operation instruction. The controlled mobile terminal can receive the behavior data converted by the operation instruction, and convert the behavior data into an operation on the controlled mobile terminal.

In the embodiment, a remote assistance control device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments, and has not been described again. As used below, the term "module" may implement a combination of software and/or hardware of a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware, is also possible and contemplated.

FIG. 3 is a structural block diagram of a remote assistance control apparatus 1 according to an embodiment of the present invention. As shown in FIG. 3, the apparatus is applied to a control mobile terminal side, and includes a first setup module 32, a display module 34, and an assist module 36. The device will be described below.

The first establishing module 32 is configured to establish a connection with the controlled mobile terminal, and the display module 34 is connected to the first establishing module 32, and configured to receive the screen display information carried by the controlled mobile terminal and carrying the assisted content, and The display information is displayed on the screen of the control mobile terminal; the assisting module 36 is connected to the display module 34, and is configured to send an operation instruction for operating the screen display information displayed on the screen of the control mobile terminal according to the assistance content. The controlled mobile terminal uses the operation instruction to remotely assist the controlled mobile terminal.

FIG. 4 is a structural block diagram of a first establishing module 32 in a remote assistance control apparatus according to an embodiment of the present invention. As shown in FIG. 4, the first establishing module 32 includes a first establishing unit 42. The setup module 32 is described.

The first establishing unit 42 is configured to establish, by the server, a channel for forwarding data between the controlled mobile terminal, including: establishing a peer P2P connection and/or server message forwarding between the mobile terminal and the controlled mobile terminal. At least one of the following channels: protocol command data channel, real-time screen image differential data channel, voice data channel.

FIG. 5 is a block diagram of a preferred structure of a remote assistance control device according to an embodiment of the present invention. As shown in FIG. 5, the device includes a first receiving module 52 and a determining module 54 in addition to all the modules shown in FIG. And initiating module 56, the device will be described below.

The first receiving module 52 is configured to receive a call request of the controlled mobile terminal sent by the server; the determining module 54 is connected to the first receiving module 52, and is configured to determine whether the controlled mobile terminal that initiates the call is to remotely assist the mobile terminal. The terminal in the list is connected to the foregoing determining module 54 and the first establishing module 32, and is configured to initiate a request for remotely assisting the controlled mobile terminal to the server if the determination result of the determining module 54 is YES.

FIG. 6 is a structural block diagram of a display module 34 in a remote assist control device according to an embodiment of the present invention. As shown in FIG. 6, the display module 34 includes a receiving unit 62 and a display unit 64. Description.

The receiving unit 62 is configured to receive the screen display information of the controlled mobile terminal that carries the assisted content and is subjected to the differential processing sent by the controlled mobile terminal; the display unit 64 is connected to the receiving unit 62, and is configured to receive the The screen display information is parsed, and the parsed screen display information is displayed on the screen of the control mobile terminal.

7 is a structural block diagram of a remote assistance control device 2 according to an embodiment of the present invention. As shown in FIG. 7, the device is applied to a controlled mobile terminal side, and includes a second establishing module 72, a transmitting module 74, and a second receiving module. 76 and operation module 78, the device will be described below.

The second establishing module 72 is configured to establish a connection with the control mobile terminal; the sending module 74 is connected to the second establishing module 72, and is configured to send the screen display information carrying the assisted content to the control mobile terminal; the second receiving module 76, connected to the sending module 74, configured to receive an operation instruction that controls the mobile terminal to operate on the screen display information of the controlled mobile terminal according to the assisted content; the operation module 78 is connected to the second receiving module 76, and is configured to The controlled mobile terminal is operated according to the operation instruction.

FIG. 8 is a structural block diagram of a second establishing module 72 in the remote assisted control device 2 according to an embodiment of the present invention. As shown in FIG. 8, the second establishing module 72 includes a second establishing unit 82. The second establishing module 72 will be described below.

The second establishing unit 82 is configured to establish and control a channel for forwarding data between the mobile terminal by using the server, including: establishing, by using the peer P2P connection between the controlled mobile terminal and the controlling mobile terminal, and/or server message forwarding as follows At least one of the channels: protocol command data channel, real-time screen image differential data channel, voice data channel.

FIG. 9 is a structural block diagram of a transmitting module 74 in a remote assisted control device 2 according to an embodiment of the present invention. As shown in FIG. 9, the transmitting module 74 includes a processing unit 92 and a transmitting unit 94, and the transmitting module 74 is performed below. Description.

The processing unit 92 is configured to perform differential processing on the screen display information; the transmitting unit 94 is connected to the processing unit 92, and is configured to send the screen display information that has undergone the differential processing to the control mobile terminal.

The present invention will be described below by taking an Android mobile phone as an example in conjunction with a specific embodiment.

As an example, the problem that the remote assistance cannot be performed between the mobile terminals in the related art is solved. In the embodiment of the present invention, a remote assistance scheme based on the Android mobile phone is proposed, which enables the mobile phone user to obtain the operation of other mobile phone users at any time. Various assistance to solve the difficulties of Android mobile phone users, providing users with various conveniences in use, improving the usage rate of Android mobile phone functions, and expanding the user population and number of Android mobile phone users.

In the embodiment of the present invention, the method may be implemented by using two entities: an Android mobile phone client software, which is installed on an Android mobile terminal device and has root authority, and is responsible for the control terminal (the same control as above) All services of the mobile terminal) and the controlled terminal (same mobile terminal as described above) are implemented. Entity 2 is server-side software deployed on a WAN server to provide a P2P implementation.

In the embodiment of the present invention, the following technical solutions are mainly adopted:

First, the client software design:

The client software is divided into two main modules:

1. The control terminal module (the same as the above control mobile terminal device);

2. The controlled terminal module (same as the above-mentioned controlled mobile terminal device);

3. Voice call module;

The control terminal module is mainly responsible for: processing the control assistance request, verifying the request information through the server, connecting the controlled machine, processing the information sent by the controlled machine, generating a control protocol command, and transmitting the control protocol command to the controlled terminal. The controlled terminal is mainly responsible for initiating the control assistance request, connecting the control terminal through the server, performing differential processing on the display content of the local screen, transmitting the screen differential content to the control terminal, and parsing the control command and applying to the local machine. The voice call module is mainly responsible for two-way voice interaction.

Second, the server-side software design:

The server software is mainly divided into three main modules: 1. P2P hole punching module; 2. Message forwarding module; 3. Business logic module.

Among them, the P2P hole punching module is mainly responsible for the direct point-to-point connection between the control terminal mobile phone and the controlled mobile phone. The message forwarding module is mainly responsible for: when some mobile phones, the control terminal and the controlled terminal are all under the symmetry router, and cannot be mutually When the peer-to-peer connection is completed through P2P hole punching, the control assistance service is completed by the message instruction forwarding mode. The business logic module is primarily responsible for controlling the functions required to assist the business.

Third, assist in the process design:

The process is as follows:

The controlled terminal (the same controlled mobile terminal as above) initiates an assistance request to register to the server, and makes a call to the control mobile phone (the same as the above-mentioned control mobile terminal). Once the phone is connected, it will be hung up immediately.

Once the control terminal receives the call that hangs up immediately after the connection, it first queries whether there is such a phone number in the local remote assistance list. If not, the process ends; if it exists, the server is queried whether there is currently a request for assistance for that number.

If the result of the server query is: 1. If there is a request for assistance, the request is taken from the server, and then the connection is established with the controlled terminal through the server. 2. If there is no assistance request, the process ends.

After the connection is established, the protocol handshake is first performed, and then the controlled terminal establishes a voice channel with the control terminal, and finally establishes a protocol data channel.

The controlled terminal sends the screen processing of the machine to the control terminal in real time, and listens to the protocol information sent by the control terminal. The protocol information will be parsed and applied to the local operation.

The control terminal combines the received on-screen information analysis and displays it on the local screen. At the same time, the screen operation of the unit is monitored, and all related operations are converted into protocol commands and sent to the controlled terminal.

Both parties can communicate with each other through the voice data channel while operating the controlled mobile phone.

After either party initiates a disconnect, the process ends.

In the following, an Android mobile phone is taken as an example, and the implementation of the technical solution is described in detail with reference to the accompanying drawings.

Install the client software as root and configure it on the Android phone, and ensure that it is always powered on.

Install the server-side software on the server and complete the configuration.

Select and configure the phone trust list through the client software.

The specific implementation steps are as follows:

FIG. 10 is an interaction diagram of a first connection between a control end and a controlled end according to an embodiment of the present invention. As shown in FIG. 10, the process of the first connection is as follows:

In step S1002, the controlled terminal initiates a request for registration to the server, and initiates a remote assistance request if the registration is successful.

In step S1004, the controlled terminal clicks on the client software, and the control terminal mobile phone number selection list appears. The user selects someone's number from them to send a controlled request. The accused request will be sent to the server first. When the server receives the controlled request, it will parse the request protocol and register the relevant information to the server to receive the received control. Subsequently, the client accepting the initiated request is notified. After receiving the notification of the server acceptance, the controlled terminal will make a call to the control terminal mobile phone and hang up immediately after being connected.

Step S1006: The client software daemon of the control terminal monitors such an hang up call, first queries whether the local control assistance phone list contains such a phone number (the number can be manually added in the software by the user, or through the user The frequency of contact with this number is automatically added). If it does not exist, ignore it. If it exists, it initiates a query to the server to query the host. The request protocol package will contain the necessary information.

After receiving the request, the server searches for the presence of the controlled request in the local database according to the information in the request packet, and returns it to the control terminal according to the actual situation.

The console receives a reply from the server and parses it. If there is no request from the host, it will be ignored. If there is a request from the host, the protocol packet for receiving the request is initiated to the server.

After receiving the receiving request, the server determines whether the permission is allowed through the content of the protocol packet information, and sends the result to the control terminal.

The console gets a reply. If not allowed, the conversation ends. If allowed, initiate a connection request to the server.

In step S1008, the server establishes data channels of both parties according to the information provided by the two parties. For the establishment of the data channel, first consider the P2P connection.

In step S1010, if the P2P connection is unsuccessful, the channel is established by server message forwarding. After the establishment is completed, notify both parties. At this point, the two sides began to control the assistance operation.

FIG. 11 is an interaction diagram of the control end and the controlled end control assistance according to an embodiment of the present invention. As shown in FIG. 11, the connection control operation assisting operation interaction process is as follows:

The connection between the control end and the controlled end is established through the server. Once the connection is established, the controlled end and the control end can communicate through voice in real time.

The controlled terminal performs differential processing on the screen image of the local machine in real time and sends it to the control terminal.

After the control terminal obtains the screen differential data of the controlled terminal, it is processed and displayed on its own screen.

When the user on the control side touches the display area, the client program sends the touched behavior data to the controlled terminal.

After the host obtains these behavior data packets, it resolves and applies it to the local machine.

During the process, either party can end the control assistance process.

Through the above scheme, the following effects can be achieved:

Any Android phone that implements the solution on the WAN can operate on another Android phone under the permission.

The operation of another Android phone is like operating a local phone. The controlled mobile phone will transmit the screen image to the control terminal mobile phone in real time, and the control mobile phone screen will display the screen on its own screen. Control The user of the terminal only needs to operate the other mobile phone as if it were a local mobile phone. The operator sees the same interface as the operator, and both parties can operate the controlled phone at the same time.

While operating on another Android mobile phone, it can also perform voice interaction with the controlled mobile phone user through the data channel, for prompting and guiding the other party.

The connection between mobile phones adopts P2P technology to improve the interaction speed and reduce the burden on the server.

The transfer picture between mobile phones adopts differential technology, which only transmits the changed parts, reduces the data flow transmitted, and improves the interaction speed.

There is a security mechanism that allows only trusted people to assist in the operation of their mobile phones.

It will be apparent to those skilled in the art that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein. The steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps thereof are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software.

The above description is only the preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

DRAWINGS

As described above, the above embodiments and preferred embodiments solve the problem that remote assistance cannot be performed between mobile terminals existing in the related art, thereby achieving the effect of realizing remote assistance between mobile terminals and improving user experience.

Claims

A remote assistance control method includes:

Controlling the mobile terminal to establish a connection with the controlled mobile terminal;

Receiving the screen display information carried by the controlled mobile terminal and carrying the assisted content, and displaying the screen display information on the screen of the control mobile terminal;

And sending, according to the assistance content, an operation instruction for operating the screen display information displayed on the screen of the control mobile terminal to the controlled mobile terminal, and using the operation instruction to the controlled mobile terminal Remote assistance.
The method of claim 1, wherein the controlling the mobile terminal to establish a connection with the controlled mobile terminal comprises:

Establishing, by the server, a channel for forwarding data with the controlled mobile terminal, comprising: establishing, by using the peer, a peer P2P connection and/or server message forwarding between the controlled mobile terminal and the controlled mobile terminal At least one:

Protocol command data channel, real-time screen image differential data channel, voice data channel.
The method of claim 1, wherein before the controlling the mobile terminal establishes a connection with the controlled mobile terminal, the method further comprises:

Receiving a call request of the controlled mobile terminal sent by the server;

Determining whether the controlled mobile terminal that initiates the call is the terminal in the remote assistance list of the control mobile terminal;

In case the determination result is yes, a request for remotely assisting the controlled mobile terminal is initiated to the server.
The method of claim 1, wherein receiving the screen display information carried by the controlled mobile terminal and carrying the assisted content, and displaying the screen display information on the screen of the controlling mobile terminal comprises:

Receiving, by the controlled mobile terminal, the screen display information that carries the assisted content and has undergone differential processing;

And the received screen display information is parsed, and the parsed screen display information is displayed on the screen of the control mobile terminal.
The method of claim 1 wherein said operational instructions comprise at least one of:

Voice operation instructions, gesture touch operation instructions, key operation instructions.
A remote assistance control method includes:

The controlled mobile terminal establishes a connection with the mobile terminal;

Sending on-screen information carrying the assisted content to the control mobile terminal;

Receiving, by the control mobile terminal, an operation instruction for operating on the screen display information of the controlled mobile terminal according to the assisted content;

And operating the controlled mobile terminal according to the operation instruction.
The method of claim 6, wherein the establishing, by the controlled mobile terminal, the connection with the controlling the mobile terminal comprises:

Establishing, by the server, a channel for forwarding data between the controlling mobile terminal, comprising: establishing, by using at least a peer P2P connection and/or server message forwarding between the controlled mobile terminal and the controlling mobile terminal, a channel as follows one:

Protocol command data channel, real-time screen image differential data channel, voice data channel.
The method of claim 6, wherein the transmitting the display information carrying the assisted content to the controlling mobile terminal comprises:

Performing differential processing on the screen display information;

The screen display information after the differential processing is sent to the control mobile terminal.
The method of claim 6 wherein said operational instructions comprise at least one of:

Voice operation instructions, gesture touch operation instructions, key operation instructions.
A remote assistance control device for controlling a mobile terminal side includes:

a first establishing module, configured to establish a connection with the controlled mobile terminal;

a display module, configured to receive the screen display information carried by the controlled mobile terminal and carry the assisted content, and display the on-screen display information on the screen of the control mobile terminal;

The assistance module is configured to send an operation instruction for operating the on-screen display information displayed on the screen of the control mobile terminal to the controlled mobile terminal according to the assistance content, by using the operation instruction The controlled mobile terminal performs remote assistance.
The apparatus of claim 10, wherein the first establishing module comprises:

a first establishing unit, configured to establish, by the server, a channel for forwarding data with the controlled mobile terminal, including: controlling, by the controlling mobile terminal, a peer P2P connection between the mobile terminal and the controlled mobile terminal Or server message forwarding establishes at least one of the following channels:

Protocol command data channel, real-time screen image differential data channel, voice data channel.
The method of claim 10, further comprising:

a first receiving module, configured to receive a call request of the controlled mobile terminal sent by the server;

a determining module, configured to determine whether the controlled mobile terminal that initiates the call is the terminal in the remote assistance list of the controlling mobile terminal;

The initiating module is configured to initiate a request for remotely assisting the controlled mobile terminal to the server if the determination result of the determining module is yes.
The device of claim 10, wherein the display module comprises:

a receiving unit, configured to receive the screen display information that is sent by the controlled mobile terminal and that carries the assisted content and has undergone differential processing;

The display unit is configured to perform parsing processing on the received screen display information, and display the parsed screen display information on a screen of the control mobile terminal.
The apparatus of claim 10, wherein the operational instruction comprises at least one of the following instructions:

Voice operation instructions, gesture touch operation instructions, key operation instructions.
A remote assistance control device is applied to the controlled mobile terminal side, including:

a second establishing module, configured to establish and connect to control the mobile terminal;

a sending module, configured to send the screen display information carrying the assisted content to the control mobile terminal;

a second receiving module, configured to receive an operation instruction that is sent by the control mobile terminal to operate on the screen display information of the controlled mobile terminal according to the assistance content;

The operation module is configured to operate the controlled mobile terminal according to the operation instruction.
The apparatus of claim 15, wherein the second establishing module comprises:

a second establishing unit, configured to establish, by the server, a channel for forwarding data between the controlled mobile terminal, including: a peer P2P connection between the controlled mobile terminal and the controlling mobile terminal, and/or Server message forwarding establishes at least one of the following channels:

Protocol command data channel, real-time screen image differential data channel, voice data channel.
The apparatus of claim 15, wherein the transmitting module comprises:

a processing unit configured to perform differential processing on the display information;

The sending unit is configured to send the screen display information that has undergone the differential processing to the control mobile terminal.
The apparatus of claim 15, wherein the operation instruction comprises at least one of the following instructions:

Voice operation instructions, gesture touch operation instructions, key operation instructions.