WO2022052649A1

WO2022052649A1 - Trajectory display method, first terminal, second terminal, and storage medium

Info

Publication number: WO2022052649A1
Application number: PCT/CN2021/107840
Authority: WO
Inventors: 杜梅霞; 尹宾
Original assignee: Oppo广东移动通信有限公司
Priority date: 2020-09-08
Filing date: 2021-07-22
Publication date: 2022-03-17
Also published as: CN112153619A

Abstract

Disclosed in the embodiments of the present application are a trajectory display method, a first terminal, a second terminal, and a storage medium. The method applied to the first terminal comprises: under the condition of establishing a wireless connection with a second terminal, sharing an interface with the second terminal, and determining a displayed interface as a first shared interface; under the condition that a trajectory display function is enabled, collecting operation information for the first shared interface, and determining an operation trajectory on the basis of the operation information; and transmitting the operation trajectory to the second terminal to allow for the second terminal to display the operation trajectory on a displayed second shared interface identical to the first shared interface.

Description

Track display method, first terminal, second terminal and storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on the application number 202010932667.0, the application date is September 08, 2020, and the application name is "A trajectory display method, a first terminal, a second terminal and a storage medium" Chinese patent application, and requires the Chinese patent application Priority, the entire content of this Chinese patent application is hereby incorporated by reference into this application.

technical field

The embodiments of the present application relate to the field of communications technologies, and in particular, to a method for displaying a trajectory, a first terminal, a second terminal, and a storage medium.

Background technique

With the continuous development of communication technologies, terminals can provide more and more functions. For example, screen sharing can be performed between terminals, as well as remote assistance.

Currently, in the process of screen sharing or remote assistance between terminals, one terminal needs to share its own display interface with another terminal. However, in the screen sharing scenario, the shared terminal can only display the operation results of the other terminal on the shared interface, and in the remote assistance scenario, the assisted terminal can only display the other terminal's own interface. The user can only know the operation result of the interface, and the interface display effect is not good.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a trajectory display method, a first terminal, a second terminal, and a storage medium, which can provide a display interface for a shared terminal or an assisted terminal during screen sharing or remote assistance between terminals , which improves the interface display effect.

The technical solutions of the embodiments of the present application are implemented as follows:

An embodiment of the present application provides a trajectory display method, which is applied to a first terminal, and the method includes:

In the case of establishing a wireless connection with the second terminal, perform interface sharing with the second terminal, and determine the displayed interface as the first shared interface;

In the case of enabling the trajectory display function, collecting operation information for the first shared interface, and determining an operation trajectory based on the operation information;

The operation track is transmitted to the second terminal, so that the second terminal can display the operation track on the displayed second sharing interface that is the same as the first sharing interface.

The embodiment of the present application provides a trajectory display method, which is applied to a second terminal, and the method includes:

In the case of establishing a wireless connection with the first terminal, perform interface sharing with the first terminal, and determine the displayed interface as the second shared interface;

receiving the operation track on the first sharing interface transmitted by the first terminal; the first sharing interface is the same interface displayed by the first terminal as the second sharing interface;

The operation track is displayed on the second sharing interface.

An embodiment of the present application provides a first terminal, including:

a first sharing module, configured to perform interface sharing with the second terminal when a wireless connection is established with the second terminal, and determine the displayed interface as the first shared interface;

a determining module, configured to collect operation information for the first shared interface when the trajectory display function is turned on, and determine an operation trajectory based on the operation information;

The first communication module is configured to transmit the operation track to the second terminal, so that the second terminal can display the operation track on the displayed second sharing interface that is the same as the first sharing interface.

An embodiment of the present application provides a second terminal, including:

a second sharing module, configured to perform interface sharing with the first terminal when a wireless connection is established with the first terminal, and determine the displayed interface as the second shared interface;

a second communication module, configured to receive the operation track on the first sharing interface transmitted by the first terminal; the first sharing interface is the same interface displayed by the first terminal as the second sharing interface;

A display module configured to display the operation track on the second sharing interface.

An embodiment of the present application provides a first terminal, including: a first processor, a first memory, a first display, and a first communication bus;

the first communication bus configured to implement a communication connection between the first processor, the first memory and the first display;

The first processor and the first display are configured to execute the first track display program stored in the first memory, so as to implement the above track display method applied to the first terminal.

An embodiment of the present application provides a second terminal, including: a second processor, a second memory, a second display, and a second communication bus;

the second communication bus configured to implement a communication connection between the second processor, the second memory and the second display;

The second processor and the second display are configured to execute the second track display program stored in the second memory, so as to implement the above track display method applied to the second terminal.

An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the above-mentioned trajectory display method.

Description of drawings

FIG. 1 is a schematic flowchart 1 of a method for displaying a trajectory according to an embodiment of the present application;

FIG. 2 is a schematic diagram 1 of display of an exemplary first terminal according to an embodiment of the present application;

FIG. 3 is a second schematic diagram of display of an exemplary first terminal according to an embodiment of the present application;

FIG. 4 is a third schematic diagram of display of an exemplary first terminal provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of an exemplary trajectory display provided by an embodiment of the present application;

FIG. 6 is a second schematic flowchart of a method for displaying a trajectory according to an embodiment of the present application;

FIG. 7 is a schematic diagram of terminal interaction in an exemplary remote assistance scenario provided by an embodiment of the present application;

FIG. 8 is a schematic structural diagram 1 of a first terminal according to an embodiment of the present application;

FIG. 9 is a second schematic structural diagram of a first terminal according to an embodiment of the present application;

FIG. 10 is a schematic structural diagram 1 of a second terminal according to an embodiment of the present application;

FIG. 11 is a second schematic structural diagram of a second terminal according to an embodiment of the present application.

detailed description

In a first aspect, an embodiment of the present application provides a trajectory display method, which is applied to a first terminal, and the method includes:

In the above method, the performing interface sharing with the second terminal and determining the displayed interface as the first shared interface includes:

determining the currently displayed first display interface as the first shared interface;

The first interface information corresponding to the first sharing interface is transmitted to the second terminal, so that the second terminal uses the first interface information to display the second sharing interface.

receiving second interface information corresponding to the second shared interface transmitted by the second terminal;

Perform interface display according to the second interface information to obtain the first shared interface.

In the above method, the collecting operation information for the first shared interface, and determining the operation trajectory based on the operation information, includes:

overlaying a transparent interface on the first sharing interface;

receiving a touch operation on the first shared interface through the transparent interface, and determining the operation information according to the touch operation;

The operation trajectory is generated on the transparent interface based on the operation information.

In the above method, the operation information includes a plurality of touch points, and the generating the operation track on the transparent interface based on the operation information includes:

On the transparent interface, connect the plurality of touch points in sequence according to the touch sequence to obtain at least one line segment;

A target special effect is added to the at least one line segment to obtain the operation trajectory.

In the above method, before performing interface sharing with the second terminal in the case of establishing a wireless connection with the second terminal, and determining the displayed interface as the first shared interface, the method further includes:

Send a control request to the user server;

receiving the cluster identification and authentication information generated by the user server according to the control request;

Use the authentication information to initiate a request to join the cluster corresponding to the cluster identifier to the real-time communication server, so as to join the cluster through the real-time communication server and complete the handshake connection with the second terminal that joins the cluster .

In a second aspect, an embodiment of the present application provides a method for displaying a trajectory, which is applied to a second terminal, and the method includes:

The operation track is displayed on the second sharing interface.

In the above method, performing interface sharing with the first terminal and determining the displayed interface as the second shared interface includes:

receiving first interface information corresponding to the first shared interface transmitted by the first terminal;

The interface is displayed according to the first interface information to obtain the second shared interface.

determining the currently displayed second display interface as the second shared interface;

The second interface information corresponding to the second sharing interface is transmitted to the first terminal, so that the first terminal uses the second interface information to display the first sharing interface.

In the above method, before the interface is shared with the first terminal when a wireless connection is established with the first terminal, and the displayed interface is determined as the second shared interface, the method further includes:

Receive the cluster identification and authentication information sent by the user server;

Send a request to join the cluster corresponding to the cluster identifier to the real-time communication server by using the authentication information, so as to join the cluster through the real-time communication server and complete the handshake connection with the first terminal that joins the cluster .

In a third aspect, an embodiment of the present application provides a first terminal, including:

A determination module, configured to collect operation information for the first shared interface when the trajectory display function is turned on, and determine an operation trajectory based on the operation information;

In the above-mentioned first terminal, the first sharing module is specifically configured to determine the currently displayed first display interface as the first sharing interface; and transmit the first interface information corresponding to the first sharing interface to the The second terminal is used for the second terminal to display the second sharing interface by using the first interface information.

In the above-mentioned first terminal, the first sharing module is specifically configured to receive second interface information corresponding to the second sharing interface transmitted by the second terminal; perform interface display according to the second interface information, and obtain the first sharing interface.

In the above-mentioned first terminal, the determining module is specifically configured to cover a transparent interface on the first sharing interface; receive a touch operation on the first sharing interface through the transparent interface, and The touch operation determines the operation information; and the operation track is generated on the transparent interface based on the operation information.

In a fourth aspect, an embodiment of the present application provides a second terminal, including:

In the above-mentioned second terminal, the second sharing module is specifically configured to receive first interface information corresponding to the first sharing interface transmitted by the first terminal; perform interface display according to the first interface information, and obtain the second shared interface.

In the above-mentioned second terminal, the second sharing module is specifically configured to determine the second display interface currently displayed as the second sharing interface; and transmit the second interface information corresponding to the second sharing interface to the second sharing interface. The first terminal is used for the first terminal to display the first sharing interface by using the second interface information.

In a fifth aspect, an embodiment of the present application provides a first terminal, including: a first processor, a first memory, a first display, and a first communication bus;

The first processor is configured to execute the first track display program stored in the first memory, so as to implement the above track display method applied to the first terminal.

In a sixth aspect, an embodiment of the present application provides a second terminal, including: a second processor, a second memory, a second display, and a second communication bus;

The second processor is configured to execute the second track display program stored in the second memory, so as to implement the above track display method applied to the second terminal.

In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the above-mentioned trajectory display method.

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

An embodiment of the present application provides a method for displaying a trajectory, which is implemented by a first terminal. The first terminal may be an electronic device such as a mobile phone and a tablet computer, which is not limited in this embodiment of the present application. FIG. 1 is a schematic flowchart 1 of a method for displaying a trajectory according to an embodiment of the present application. As shown in Figure 1, it mainly includes the following steps:

S101. In the case of establishing a wireless connection with the second terminal, perform interface sharing with the second terminal, and determine the displayed interface as the first shared interface.

In the embodiment of the present application, when a wireless connection is established with the second terminal, the first terminal performs interface sharing with the second terminal, and determines the displayed interface as the first shared interface.

It should be noted that, in the embodiments of the present application, the second terminal may be an electronic device such as a mobile phone and a tablet computer, which is not limited in the embodiments of the present application.

It should be noted that, in the embodiment of the present application, the first terminal needs to establish a wireless connection with the second terminal first, and then perform interface sharing with the second terminal. Specifically, the first terminal and the second terminal can simultaneously enable screen sharing or remote assistance functions, so as to access the same network for account matching and wireless connection. Certainly, other manners may also be used to establish a wireless connection between the first terminal and the second terminal, which is not limited in this embodiment of the present application.

FIG. 2 is a schematic diagram 1 of display of an exemplary first terminal according to an embodiment of the present application. As shown in FIG. 2 , in the embodiment of the present application, the user can click the “remote assistance” icon displayed on the first terminal to enable the remote assistance function, thereby further performing account connection and matching with the second terminal that also has the remote assistance function enabled. , to realize the wireless connection with the second terminal.

FIG. 3 is a second schematic diagram of display of an exemplary first terminal according to an embodiment of the present application. As shown in FIG. 3 , in the embodiment of the present application, the user can click the "screen sharing" icon displayed on the first terminal to enable the screen sharing function, thereby further performing account connection and matching with the second terminal that also enables the screen sharing function , to realize the wireless connection with the second terminal.

It can be understood that, in the embodiment of the present application, when the remote assistance or screen sharing function is enabled on the first terminal, the second terminal also needs to enable the same function.

It can be understood that, in the embodiment of the present application, when the first terminal establishes a wireless connection with the second terminal, it can share the interface with the second terminal, so that the interface displayed by itself is determined as the first shared interface. . Of course, since the terminal and the second terminal share the interface, in fact, the display interfaces of the two terminals are the same, that is, the interface displayed on the second terminal is the same interface as the first shared interface. The specific first sharing interface is not limited in this embodiment of the present application.

It should be noted that, in this embodiment of the present application, the first terminal and the second terminal share the interface, which may be the first terminal displaying the interface displayed by the second terminal, or the second terminal displaying the interface displayed by the first terminal. , both actually display the same interface.

Specifically, in the embodiment of the present application, the first terminal and the second terminal perform interface sharing, and determining the displayed interface as the first shared interface includes: determining the currently displayed first display interface as the first shared interface ; transmitting the first interface information corresponding to the first sharing interface to the second terminal, so that the second terminal can display the second sharing interface by using the first interface information.

It can be understood that, in this embodiment of the present application, in a screen sharing scenario, the first terminal may directly determine the currently displayed first display interface as the first shared interface, and assign the first shared interface corresponding to the first shared interface. The interface information is wirelessly transmitted to the second terminal, so that the second terminal can display a second shared interface identical to the first shared interface according to the first interface information.

Specifically, in the embodiment of the present application, the first terminal and the second terminal perform interface sharing, and determine the displayed interface as the first shared interface, including: receiving a second interface corresponding to the second shared interface transmitted by the second terminal interface information; performing interface display according to the second interface information to obtain a first shared interface.

It can be understood that, in the embodiment of the present application, in the case of remote assistance, the first terminal needs to display the interface displayed by the second terminal, which is the second shared interface. Therefore, the second terminal can The second interface information corresponding to the shared interface is transmitted to the first terminal, and the first terminal can display the same interface as the second shared interface according to the received second interface information, which is the first shared interface.

S102. In the case where the trajectory display function is enabled, collect operation information for the first shared interface, and determine the operation trajectory based on the operation information.

In the embodiment of the present application, when the track display function is enabled, the first terminal may collect operation information for the first shared interface, and determine the operation track based on the operation information.

It should be noted that, in this embodiment of the present application, when a wireless connection is established between the first terminal and the second terminal, for example, the first terminal and the second terminal are in the process of screen sharing or remote assistance, the user can use the A specific operation enables the trace display function. The specific enabling manner of the track display function is not limited in this embodiment of the present application.

FIG. 4 is a third schematic diagram of display of an exemplary first terminal according to an embodiment of the present application. As shown in FIG. 4 , the screen sharing function is enabled on the first terminal, and a wireless connection is established with the second terminal. The user can click the “operation track” icon on the first terminal to enable the track display function.

It can be understood that, in the embodiment of the present application, the user can perform a touch operation on the first sharing interface displayed by the first terminal. Draw a circle mark, or slide the interface. In the remote assistance scenario, the user can click on certain function entries provided in the first sharing interface. Therefore, the first terminal can collect the corresponding operation information, thereby Determine the operation track corresponding to the touch operation.

Specifically, in the embodiment of the present application, the first terminal collects operation information for the first sharing interface, and determines the operation trajectory based on the operation information, including: covering the first sharing interface with a transparent interface; The touch operation of the first shared interface, and the operation information is determined according to the touch operation; the operation track is generated on the transparent interface based on the operation information.

It can be understood that, in the embodiment of the present application, the first terminal may cover the displayed first sharing interface with a layer of transparent interface for drawing the operation track, and the transparent interface does not affect the user's click on the first sharing interface. Waiting for the touch operation, when the first terminal draws and displays the operation track on the transparent interface, the terminal may continue to receive the touch operation on the first shared interface through the transparent interface.

It should be noted that, in the embodiment of the present application, the operation information includes multiple touch points, and the first terminal generates an operation track on the transparent interface based on the operation information, including: The touch sequences are connected in sequence to obtain at least one line segment; target special effects are added to at least one line segment to obtain an operation track.

It should be noted that, in the embodiment of the present application, the first terminal may sequentially connect the collected touch points on the transparent interface, and form a line segment between every two touch points, thereby obtaining at least one touch point. Line segment, and the end point of the previous line segment is used as the start point of the next line segment, with continuity. Afterwards, the terminal may add target special effects to at least one line segment on the transparent interface, thereby obtaining the operation trajectory. Compared with directly displaying at least one line segment, the obtained operation trajectory has a better visual effect. Among them, the target special effect can include that the thickness of the line segment gradually decreases with time, so as to achieve the effect of a large beginning and a small tail, and it can also include that the transparency of the line segment gradually decreases with time to achieve the effect of gradually disappearing.

It should be noted that, in the embodiment of the present application, a target special effect may be stored in the first terminal, so that the target special effect is called during the drawing of the operation track. The specific target effect can be set according to actual needs, which is not limited in the embodiment of the present application.

S103: Transmit the operation track to the second terminal, so that the second terminal can display the operation track on the displayed second sharing interface that is the same as the first sharing interface.

In the embodiment of the present application, after obtaining the operation track, the first terminal can transmit the operation track to the second terminal for displaying the operation track on the displayed second sharing interface that is the same as the first sharing interface.

It can be understood that, in the embodiment of the present application, the first terminal and the second terminal share the interface, and the first shared interface displayed on the first terminal is the same as the second shared interface displayed on the second terminal. . The first terminal transmits the operation track on the first sharing interface to the second terminal, and the second terminal can display the operation track on the second sharing interface displayed by itself, which is equivalent to displaying the user using the first terminal for the first terminal. An operation track of the interface is shared, so that the user using the second terminal can know the operation process of the user using the first terminal, not just the operation result.

FIG. 5 is a schematic diagram of a track display of an exemplary second sharing interface provided by an embodiment of the present application. As shown in FIG. 5 , in the embodiment of the present application, the second terminal may display the operation track on the second sharing interface according to the received operation track. It should be noted that the first sharing interface on the first terminal is the same as the second sharing interface, and the first terminal draws an operation track on the transparent interface on the first sharing interface, that is, the content actually displayed by the first terminal is the same as the first sharing interface. The content displayed on the two terminals is the same.

It should be noted that, in the embodiments of the present application, in the scenario of remote assistance, the first terminal also needs to transmit the operation information to the second terminal, so that the second terminal displays the trajectory on the second sharing interface. , and can also respond accordingly on the second sharing interface according to the operation information.

It should be noted that, in the embodiments of the present application, the first terminal is a terminal that provides screen sharing or a terminal that provides remote assistance, and correspondingly, the second terminal is a shared terminal or an assisted terminal.

It should be noted that, in the embodiment of the present application, in the case of establishing a wireless connection with the second terminal, before performing interface sharing with the second terminal and determining the displayed interface as the first shared interface, the method further includes: Send a control request to the user server; receive the cluster identification and authentication information generated by the user server according to the control request; use the authentication information to initiate a request to join the cluster corresponding to the cluster identification to the real-time communication server, so as to join the cluster through the real-time communication server, and complete Handshake connection with the second terminal joining the cluster.

It should be noted that, in the embodiment of the present application, the first terminal and the second terminal complete a handshake connection, so as to mutually determine a wireless connection mode for wireless connection. In a scenario where a direct connection can be established between the first terminal and the second terminal, the first terminal and the second terminal can directly exchange information with each other. For example, in the case where the first terminal remotely assists the second terminal, the first terminal can The operation track is transmitted to the second terminal through the data channel. In addition, in a scenario where a direct connection cannot be established between the first terminal and the second terminal, the first terminal and the second terminal can exchange information through the relay server. For example, in the case where the first terminal remotely assists the second terminal, the first terminal The terminal transmits the operation track to the second terminal through the relay server. A specific manner in which the first terminal and the second terminal perform information interaction is not limited in this embodiment of the present application.

An embodiment of the present application provides a method for displaying a trajectory, which is applied to a first terminal, including: in the case of establishing a wireless connection with a second terminal, performing interface sharing with the second terminal, and determining the displayed interface as the first sharing interface; when the trajectory display function is turned on, collect operation information for the first shared interface, and determine the operation trajectory based on the operation information; transmit the operation trajectory to the second terminal for the second terminal to share with the first terminal on the display The operation track is displayed on the second shared interface with the same interface. The trajectory display method provided by the embodiments of the present application can provide the shared terminal or the assisted terminal with the operation trajectory of the display interface during the process of screen sharing or remote assistance between terminals, thereby improving the interface display effect.

An embodiment of the present application provides a method for displaying a trajectory, which is applied to a second terminal. FIG. 6 is a second schematic flowchart of a method for displaying a trajectory according to an embodiment of the present application. As shown in Figure 6, it mainly includes the following steps:

S601. In the case of establishing a wireless connection with the first terminal, perform interface sharing with the first terminal, and determine the displayed interface as the second shared interface.

In the embodiment of the present application, when a wireless connection is established with the first terminal, the second terminal performs interface sharing with the first terminal, and determines the displayed interface as the second shared interface.

It should be noted that, in the embodiment of the present application, the second terminal needs to establish a wireless connection with the first terminal first, and then perform interface sharing with the first terminal. Specifically, the second terminal and the first terminal can enable screen sharing or remote assistance functions at the same time, so as to access the same network for account matching and wireless connection. Certainly, other manners may also be used to establish a wireless connection between the second terminal and the first terminal, which is not limited in this embodiment of the present application.

It should be noted that, in the embodiment of the present application, the second terminal shares the interface with the first terminal, which may be the second terminal displaying the interface displayed by the first terminal, or the first terminal displaying the interface displayed by the second terminal. , the two actually display the same interface.

Specifically, in the embodiment of the present application, the second terminal performs interface sharing with the first terminal, and determines the displayed interface as the second shared interface, including: receiving a first interface corresponding to the first shared interface transmitted by the first terminal interface information; performing interface display according to the first interface information to obtain a second shared interface.

Specifically, in the embodiment of the present application, the second terminal performs interface sharing with the first terminal, and determining the displayed interface as the second shared interface includes: determining the currently displayed second display interface as the second shared interface ; transmitting the second interface information corresponding to the second sharing interface to the first terminal, so that the first terminal can display the first sharing interface by using the second interface information.

S602. Receive the operation track on the first sharing interface transmitted by the first terminal; the first sharing interface is the same interface displayed by the first terminal as the second sharing interface.

In the embodiment of the present application, the second terminal may receive the operation track on the first sharing interface transmitted by the first terminal, where the first sharing interface is the same interface displayed by the first terminal as the second sharing interface.

It can be understood that, in the embodiment of the present application, the second terminal shares the interface with the first terminal, therefore, the interfaces displayed by the two are actually the same, that is, the second shared interface displayed by the second terminal The first sharing interface displayed by the first terminal is the same, and the only difference is that the displayed terminals are different. A user using the first terminal can perform a touch operation on the first shared interface displayed by the first terminal, so that the first terminal can obtain the operation track on the first shared interface and transmit it to the second terminal.

S603. Display the operation track on the second sharing interface.

In the embodiment of the present application, after receiving the operation track transmitted by the first terminal, the second terminal may display the operation track on the displayed second sharing interface.

It can be understood that, in the embodiment of the present application, the second terminal shares the interface with the first terminal, and the second shared interface displayed on the second terminal is the same as the first shared interface displayed on the first terminal. . Therefore, when the second terminal receives the operation track on the first shared interface, it can use the operation track to display the track on the second shared interface displayed by itself, which is equivalent to two terminals displaying the same interface, and On the user's operation track on the same interface, the user using the second terminal can learn the operation process of the user using the first terminal, not just the operation result.

It should be noted that, in the embodiment of the present application, in the scenario of remote assistance, the first terminal may also transmit operation information for the first shared interface to the second terminal, that is, the second terminal may also receive the operation information, Therefore, when the second terminal displays the operation track on the second sharing interface, it can also make a corresponding response on the second sharing interface according to the operation information.

It should be noted that, in the embodiment of the present application, when the second terminal establishes a wireless connection with the first terminal, the second terminal performs interface sharing with the first terminal, and before determining the displayed interface as the second shared interface, further Perform the following steps: receive the cluster identification and authentication information sent by the user server; use the authentication information to send a request for joining the cluster corresponding to the cluster identification to the real-time communication server, so as to join the cluster through the real-time communication server, and complete the first step with joining the cluster. Terminal handshake connection.

It should be noted that, in the embodiment of the present application, the second terminal and the first terminal complete a handshake connection, so as to mutually determine a wireless connection mode for wireless connection. In the scenario where the first terminal and the second terminal can establish a direct connection, the first terminal and the second terminal can directly exchange information with each other. For example, when the first terminal remotely assists the second terminal, the second terminal can The information of its own display interface is transmitted to the first terminal through the audio and video channel. In addition, in a scenario where a direct connection cannot be established between the first terminal and the second terminal, the first terminal and the second terminal can exchange information through the relay server. For example, in the case where the first terminal remotely assists the second terminal, the second terminal The terminal transmits the information of its own display interface to the first terminal through the relay server. A specific manner in which the first terminal and the second terminal perform information interaction is not limited in this embodiment of the present application.

FIG. 7 is a schematic diagram of terminal interaction in an exemplary remote assistance scenario provided by an embodiment of the present application. As shown in FIG. 7 , in the scenario where the first terminal remotely assists the second terminal, the first terminal first requests assistance/control from the user server, and then the user server generates a unified room identifier and authentication token, wherein the room identifier That is, it is equivalent to the cluster identification, and the authentication token is equivalent to the authentication information. The user server can send the room identification and authentication token to the first terminal and the second terminal, and the first terminal and the second terminal can initiate connection authentication to the real-time communication server according to the received room identification and authentication token, After joining the room corresponding to the room identifier, the real-time communication server can complete the handshake connection between the first terminal and the second terminal through the SDP protocol exchange. After that, in the scenario where the first terminal and the second terminal are directly connected, the second terminal can transmit the information of the display interface to the first terminal through the audio and video channel, and the first terminal can transmit the operation information to the second terminal through the data channel In the scenario where the first terminal and the second terminal are connected via relay, the second terminal sends the information of the display interface to the first terminal through the relay server, and the first terminal sends the operation information to the second terminal through the relay server.

Embodiments of the present application provide a first terminal. FIG. 8 is a schematic structural diagram 1 of a first terminal according to an embodiment of the present application. As shown in Figure 8, the first terminal includes:

A first sharing module 801, configured to perform interface sharing with the second terminal when a wireless connection is established with the second terminal, and determine the displayed interface as the first shared interface;

The determining module 802 is configured to collect operation information for the first shared interface under the condition of enabling the trajectory display function, and determine an operation trajectory based on the operation information;

The first communication module 803 is configured to transmit the operation track to the second terminal, so that the second terminal can display the operation track on the displayed second sharing interface that is the same as the first sharing interface .

In an embodiment of the present application, the first sharing module 801 is specifically configured to determine the currently displayed first display interface as the first sharing interface; and transmit the first interface information corresponding to the first sharing interface to the second terminal, so that the second terminal uses the first interface information to display the second sharing interface.

In an embodiment of the present application, the first sharing module 801 is specifically configured to receive second interface information corresponding to the second sharing interface transmitted by the second terminal; and perform interface display according to the second interface information to obtain the first sharing interface.

In an embodiment of the present application, the determining module 802 is specifically configured to cover a transparent interface on the first sharing interface; receive a touch operation on the first sharing interface through the transparent interface, and according to The touch operation determines the operation information; and the operation track is generated on the transparent interface based on the operation information.

In an embodiment of the present application, the determining module 802 is specifically configured to, on the transparent interface, connect the plurality of touch points in sequence according to the touch sequence to obtain at least one line segment; Add target special effects to obtain the operation trajectory.

In an embodiment of the present application, the first communication module 803 is further configured to send a control request to a user server; receive a cluster identifier and authentication information generated by the user server according to the control request; use the authentication The information initiates a request to join the cluster corresponding to the cluster identifier to the real-time communication server, so as to join the cluster through the real-time communication server and complete the handshake with the second terminal that joins the cluster.

FIG. 9 is a second schematic structural diagram of a first terminal according to an embodiment of the present application. As shown in FIG. 9 , the first terminal includes: a first processor 901, a first memory 902, a first display 903 and a first communication bus 904;

The first communication bus 904 is configured to realize the communication connection between the first processor 901, the first memory 902 and the first display 903;

The first processor 901 is configured to execute the first track display program stored in the first memory 902, so as to implement the above track display method applied to the first terminal.

The embodiment of the present application provides a second terminal. FIG. 10 is a schematic structural diagram 1 of a second terminal according to an embodiment of the present application. As shown in Figure 10, the second terminal includes:

The second sharing module 1001 is configured to perform interface sharing with the first terminal when a wireless connection is established with the first terminal, and determine the displayed interface as the second shared interface;

The second communication module 1002 is configured to receive the operation track on the first sharing interface transmitted by the first terminal; the first sharing interface is the same interface displayed by the first terminal as the second sharing interface;

The display module 1003 is configured to display the operation track on the second sharing interface.

In an embodiment of the present application, the second sharing module 1001 is specifically configured to receive first interface information corresponding to the first sharing interface transmitted by the first terminal; and perform interface display according to the first interface information to obtain the second sharing interface.

In an embodiment of the present application, the second sharing module 1001 is specifically configured to determine the second display interface currently displayed as the second sharing interface; and transmit the second interface information corresponding to the second sharing interface to the first terminal, so that the first terminal uses the second interface information to display the first sharing interface.

In an embodiment of the present application, the second communication module 1002 is further configured to receive the cluster identifier and authentication information sent by the user server; and use the authentication information to send the real-time communication server to join the cluster corresponding to the cluster identifier request to join the cluster through the real-time communication server, and complete the handshake connection with the first terminal that joins the cluster.

FIG. 11 is a second schematic structural diagram of a second terminal according to an embodiment of the present application. As shown in FIG. 11, in the embodiment of the present application, the second terminal includes: a second processor 1101, a second memory 1102, a second display 1103, and a second communication bus 1104;

The second communication bus 1104 is configured to realize the communication connection between the second processor 1101, the second memory 1102 and the second display 1103;

The second processor 1101 is configured to execute the second track display program stored in the second memory 1102, so as to implement the above track display method applied to the second terminal.

An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the above-mentioned trajectory display method. The computer-readable storage medium may be a volatile memory (volatile memory), such as a random access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (Read-only memory) Only Memory, ROM), flash memory (flash memory), hard disk (Hard Disk Drive, HDD) or solid-state drive (Solid-State Drive, SSD); it can also be a respective device including one or any combination of the above memories, such as Mobile phones, computers, tablet devices, personal digital assistants, etc.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including but not limited to disk storage, optical storage, and the like.

The present application is described with reference to schematic flowcharts and/or block diagrams of implementations of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It will be understood that each process and/or block in the schematic flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the schematic flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a process or processes and/or a block or blocks in the block diagrams.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions An apparatus implements the functions specified in a flow or flows of the implementation flow diagram and/or a block or blocks of the block diagram.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the implementing flow diagram and/or the block or blocks of the block diagram.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application. , shall be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Industrial Applicability

Embodiments of the present application provide a trajectory display method, a first terminal, a second terminal, and a storage medium, and the method applied to the first terminal includes: in the case of establishing a wireless connection with the second terminal, interface with the second terminal share, and determine the displayed interface as the first shared interface; in the case of enabling the trajectory display function, collect operation information for the first shared interface, and determine the operation trajectory based on the operation information; transmit the operation trajectory to the second terminal, for the second terminal to display the operation track on the displayed second sharing interface that is the same as the first sharing interface. The technical solutions provided by the embodiments of the present application can provide the shared terminal or the assisted terminal with the operation track of the display interface during the process of screen sharing or remote assistance between terminals, thereby improving the interface display effect.

Claims

A track display method, applied to a first terminal, the method comprising:

In the case of establishing a wireless connection with the second terminal, perform interface sharing with the second terminal, and determine the displayed interface as the first shared interface;

In the case of enabling the trajectory display function, collecting operation information for the first shared interface, and determining an operation trajectory based on the operation information;

The operation track is transmitted to the second terminal, so that the second terminal can display the operation track on the displayed second sharing interface that is the same as the first sharing interface.
The method according to claim 1, wherein the performing interface sharing with the second terminal and determining the displayed interface as the first shared interface comprises:

determining the currently displayed first display interface as the first shared interface;

The first interface information corresponding to the first sharing interface is transmitted to the second terminal, so that the second terminal uses the first interface information to display the second sharing interface.
The method according to claim 1, wherein the performing interface sharing with the second terminal and determining the displayed interface as the first shared interface comprises:

receiving second interface information corresponding to the second shared interface transmitted by the second terminal;

Perform interface display according to the second interface information to obtain the first shared interface.
The method according to claim 1, wherein the collecting operation information for the first shared interface, and determining an operation trajectory based on the operation information, comprises:

overlaying a transparent interface on the first sharing interface;

receiving a touch operation on the first shared interface through the transparent interface, and determining the operation information according to the touch operation;

The operation trajectory is generated on the transparent interface based on the operation information.
The method according to claim 4, wherein the operation information includes a plurality of touch points, and the generating the operation track on the transparent interface based on the operation information comprises:

On the transparent interface, connect the plurality of touch points in sequence according to the touch sequence to obtain at least one line segment;

A target special effect is added to the at least one line segment to obtain the operation trajectory.
The method according to claim 1, wherein, in the case of establishing a wireless connection with the second terminal, performing interface sharing with the second terminal, and before determining the displayed interface as the first shared interface, the Methods also include:

Send a control request to the user server;

receiving the cluster identification and authentication information generated by the user server according to the control request;

Use the authentication information to initiate a request to join the cluster corresponding to the cluster identifier to the real-time communication server, so as to join the cluster through the real-time communication server and complete the handshake connection with the second terminal that joins the cluster .
A track display method, applied to a second terminal, the method comprising:

In the case of establishing a wireless connection with the first terminal, perform interface sharing with the first terminal, and determine the displayed interface as the second shared interface;

receiving the operation track on the first sharing interface transmitted by the first terminal; the first sharing interface is the same interface displayed by the first terminal as the second sharing interface;

The operation track is displayed on the second sharing interface.
The method according to claim 7, wherein the performing interface sharing with the first terminal and determining the displayed interface as the second shared interface comprises:

receiving first interface information corresponding to the first shared interface transmitted by the first terminal;

The interface is displayed according to the first interface information to obtain the second shared interface.
The method according to claim 7, wherein the performing interface sharing with the first terminal and determining the displayed interface as the second shared interface comprises:

determining the currently displayed second display interface as the second shared interface;

The second interface information corresponding to the second sharing interface is transmitted to the first terminal, so that the first terminal uses the second interface information to display the first sharing interface.
The method according to claim 7, wherein, in the case of establishing a wireless connection with the first terminal, performing interface sharing with the first terminal, and before determining the displayed interface as the second shared interface, the Methods also include:

Receive the cluster identification and authentication information sent by the user server;

Send a request to join the cluster corresponding to the cluster identifier to the real-time communication server by using the authentication information, so as to join the cluster through the real-time communication server and complete the handshake connection with the first terminal that joins the cluster .
A first terminal, comprising:

a first sharing module, configured to perform interface sharing with the second terminal when a wireless connection is established with the second terminal, and determine the displayed interface as the first shared interface;

a determining module, configured to collect operation information for the first shared interface when the trajectory display function is turned on, and determine an operation trajectory based on the operation information;

The first communication module is configured to transmit the operation track to the second terminal, so that the second terminal can display the operation track on the displayed second sharing interface that is the same as the first sharing interface.
The first terminal of claim 11, wherein,

The first sharing module is specifically configured to determine the currently displayed first display interface as the first sharing interface; and transmit the first interface information corresponding to the first sharing interface to the second terminal for use in the second terminal. The second terminal uses the first interface information to display the second sharing interface.
The first terminal of claim 11, wherein,

The first sharing module is specifically configured to receive second interface information corresponding to the second sharing interface transmitted by the second terminal; perform interface display according to the second interface information to obtain the first sharing interface.
The first terminal of claim 11, wherein,

The determining module is specifically configured to cover a transparent interface on the first shared interface; receive a touch operation on the first shared interface through the transparent interface, and determine the operation according to the touch operation information; and generating the operation track on the transparent interface based on the operation information.
A second terminal, comprising:

a second sharing module, configured to perform interface sharing with the first terminal when a wireless connection is established with the first terminal, and determine the displayed interface as the second shared interface;

a second communication module, configured to receive the operation track on the first sharing interface transmitted by the first terminal; the first sharing interface is the same interface displayed by the first terminal as the second sharing interface;

A display module configured to display the operation track on the second sharing interface.
The second terminal of claim 15, wherein,

The second sharing module is specifically configured to receive first interface information corresponding to the first sharing interface transmitted by the first terminal; perform interface display according to the first interface information to obtain the second sharing interface.
The second terminal of claim 15, wherein,

The second sharing module is specifically configured to determine the currently displayed second display interface as the second sharing interface; and transmit the second interface information corresponding to the second sharing interface to the first terminal for use in the first terminal. The first terminal uses the second interface information to display the first sharing interface.
A first terminal, comprising: a first processor, a first memory, a first display and a first communication bus;

the first communication bus configured to implement a communication connection between the first processor, the first memory and the first display;

The first processor is configured to execute the first track display program stored in the first memory, so as to implement the track display method according to any one of claims 1-6.
A second terminal, comprising: a second processor, a second memory, a second display and a second communication bus;

the second communication bus configured to implement a communication connection between the second processor, the second memory and the second display;

The second processor is configured to execute the second track display program stored in the second memory, so as to implement the track display method of any one of claims 7-10.
A computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the trajectory display method according to any one of claims 1-10.