CN114237473A

CN114237473A - Same-screen interaction method, device and system, storage medium and electronic device

Info

Publication number: CN114237473A
Application number: CN202111407088.5A
Authority: CN
Inventors: 单振宇; 杨凯; 邓强东; 谭文彦; 杨泽洋; 刘冬冬
Original assignee: Weimin Insurance Agency Co Ltd
Current assignee: Weimin Insurance Agency Co Ltd
Priority date: 2021-11-24
Filing date: 2021-11-24
Publication date: 2022-03-25

Abstract

The application discloses a method, a device and a system for same-screen interaction, a storage medium and an electronic device. Wherein, the method comprises the following steps: detecting a first operation on a first terminal, wherein the first operation is used for indicating to transfer the operation right of a target page from the first terminal to a second terminal, the first terminal and the second terminal are terminals for performing same-screen interaction, and the target page comprises same-screen interaction content; and responding to the first operation, sending a permission transfer instruction to the second terminal so as to allow the second terminal to operate the target page. The application solves the technical problem of low communication efficiency in the related technology.

Description

Same-screen interaction method, device and system, storage medium and electronic device

Technical Field

The application relates to the field of internet, in particular to a method, a device and a system for on-screen interaction, a storage medium and an electronic device.

Background

In many business scenarios, one-screen technologies are used, such as communication between business partners, corporate meetings located in different areas, sales and remote marketing of related products to users.

The same screen is the meaning that a plurality of mobile phones and PC computer equipment watch the screen display information of one party remotely and simultaneously, and is remote sharing of the screen, the sharing content comprises but is not limited to videos, pictures, characters and the like, and the information of the two parties in the file sharing process can be ensured to be synchronized through multiple channels of vision, hearing and touch, so that the information communication is facilitated.

In the interaction process, one party can combine writing, annotating, drawing and computer operation on shared content, and through the network interaction function, the sharing of data and annotation content in different places can be realized.

In view of the above problems, no effective solution has been proposed.

Disclosure of Invention

The embodiment of the application provides a method, a device and a system for same-screen interaction, a storage medium and an electronic device, so as to at least solve the technical problem of low communication efficiency in the related technology.

According to an aspect of an embodiment of the present application, there is provided a method for on-screen interaction, including: detecting a first operation on a first terminal, wherein the first operation is used for indicating to transfer the operation right of a target page from the first terminal to a second terminal, the first terminal and the second terminal are terminals for performing same-screen interaction, and the target page comprises same-screen interaction content; and responding to the first operation, sending a permission transfer instruction to the second terminal so as to allow the second terminal to operate the target page.

According to another aspect of the embodiments of the present application, there is also provided a method for on-screen interaction, including: receiving a permission transfer instruction sent by a first terminal, wherein the permission transfer instruction is used for the first terminal to instruct to transfer the operation right of a target page from the first terminal to a second terminal, the first terminal and the second terminal are terminals for carrying out same-screen interaction, and the target page comprises same-screen interaction content; and responding to the authority transfer instruction, and detecting the operation performed on the target page on the second terminal.

According to another aspect of the embodiments of the present application, there is also provided a method for on-screen interaction, including: displaying a target page in a display interface of a first terminal, wherein the display interface comprises a permission transfer control; receiving a first operation for the permission transfer control, wherein the first operation is used for indicating that the operation right of a target page is transferred from a first terminal to a second terminal, the first terminal and the second terminal are terminals for performing same-screen interaction, and the target page comprises same-screen interaction content; and responding to the first operation, and displaying the target page operated by the second terminal by the first terminal.

According to another aspect of the embodiments of the present application, there is also provided an on-screen interaction system, including: the first terminal is used for: sending a permission transfer instruction to a second terminal under the condition of detecting a first operation, wherein the first operation is used for indicating that the operation right of a target page is transferred from the first terminal to the second terminal, the permission transfer instruction is used for indicating that the target page is allowed to be operated on the second terminal, and the target page comprises content interacted with a screen; the second terminal is used for: and after receiving the authority transfer instruction sent by the second terminal, detecting the operation performed on the target page on the second terminal.

According to another aspect of the embodiments of the present application, there is also provided an on-screen interaction apparatus, including: the first detection unit is used for detecting a first operation on a first terminal, the first operation is used for indicating that the operation right of a target page is transferred from the first terminal to a second terminal, the first terminal and the second terminal are terminals for performing same-screen interaction, and the target page comprises same-screen interaction content; and the sending unit is used for responding to the first operation and sending the authority transfer instruction to the second terminal so as to allow the second terminal to operate the target page.

According to another aspect of the embodiments of the present application, there is also provided an on-screen interaction apparatus, including: the receiving unit is used for receiving a permission transfer instruction sent by a first terminal, the permission transfer instruction is used for the first terminal to instruct to transfer the operation right of a target page from the first terminal to a second terminal, the first terminal and the second terminal are terminals for performing same-screen interaction, and the target page comprises same-screen interaction content; and the second detection unit is used for responding to the authority transfer instruction and detecting the operation performed on the target page on the second terminal.

According to another aspect of the embodiments of the present application, there is also provided a storage medium including a stored program which, when executed, performs the above-described method.

According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the above method through the computer program.

According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the steps of any of the embodiments of the method described above.

In the embodiment of the application, in the process of performing the same-screen interaction, the screen operation behavior playback and the change on the screen of the same page can be synchronized to the screen of another terminal from one terminal in real time by the first terminal and the second terminal, and the operation right can be transferred to the second terminal where the designated user is located on the first terminal (and can be recovered according to needs in the operation process and have the capability of secondary operation).

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a schematic diagram of a hardware environment for a method of on-screen interaction according to an embodiment of the application;

FIG. 2 is a flow chart of an alternative on-screen interaction method according to an embodiment of the application;

FIG. 3 is a schematic diagram of an alternative on-screen interactive interface according to an embodiment of the present application;

FIG. 4 is a schematic diagram of an alternative on-screen interactive interface according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an alternative on-screen interactive interface according to an embodiment of the present application;

FIG. 6 is a schematic diagram of an alternative on-screen interactive interface according to an embodiment of the present application;

FIG. 7 is a schematic diagram of an alternative on-screen interactive interface according to an embodiment of the present application;

FIG. 8 is a schematic diagram of an alternative on-screen interactive interface according to an embodiment of the present application;

FIG. 9 is a schematic diagram of an alternative on-screen interactive interface according to an embodiment of the present application;

FIG. 10 is a schematic diagram of an alternative on-screen interactive interface according to an embodiment of the present application;

FIG. 11 is a schematic diagram of an alternative on-screen interaction scheme according to an embodiment of the present application;

FIG. 12 is a schematic diagram of an alternative binary tree of web page elements according to an embodiment of the present application;

FIG. 13 is a schematic diagram of an alternative on-screen interaction device in accordance with embodiments of the present application;

FIG. 14 is a schematic diagram of an alternative on-screen interaction device in accordance with embodiments of the present application; and the number of the first and second groups,

fig. 15 is a block diagram of a terminal according to an embodiment of the present application.

Detailed Description

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

The on-screen explanation in the related art is that one party plays and checks, only one party performs operation reproduction, and the operation right is not transferred in a real-time display state of the same page. Technically, one side of the screen is displayed and operated, and the screen is recorded, packaged and distributed to the other side of the screen for display and play, and the other side of the screen is operated by the other side in an interactive mode; the other technology belongs to the field of remote operation of a screen of an opposite side to assist the opposite side to complete operation, after one side A initiates a request for the other side B to control the auxiliary operation appeal of a computer, the screen of the other side B displays the screen of the opposite side A in real time, at the moment, the screen of the side B can be operated on the screen of the side B to complete the operation, but in the process, the screen of the opposite side has the real-time display and operation control right, the interchange of the two sides of the operation cannot be realized, and the real-time information display and operation synchronization of the same information flow on the two sides of the operation and the operation transfer are realized.

In addition, the current scheme does not span multiple WEB pages for screen sharing, and cannot carry out operation mutual conversion across the pages, the user A jumps to the page C, and the user B cannot receive the synchronous information of the page C. To solve the above problems, according to an aspect of the embodiments of the present application, an embodiment of a method and a system for on-screen interaction is provided. The application can meet the real-time transfer of the operation right in the same page state, and can feed back the operation condition of the page in real time, so that one-to-one operation is met, one-to-many conditions are also met, the transfer of real-time screen operation control and page change synchronous display caused by information change are technically met, the use appeal of a speaker in a complex scene can be met, the real-time synchronous demonstration of the page information can be guaranteed through demonstration change and operation of information flow, and more innovations are made on the operation display interaction process aiming at the complex use scene technically.

Alternatively, in this embodiment, the above-mentioned on-screen interaction method may be applied to a hardware environment formed by the terminal 101 and the server 103 as shown in fig. 1, where the number of terminals is at least two, and the terminal includes a first terminal having a right to manage an operation right and at least one second terminal having a use right of the operation right, during the on-screen interaction between the first terminal and the second terminal, the first terminal sends a right transfer instruction to the second terminal to indicate that an operation on the target page is permitted on the second terminal in a case of detecting a first operation (indicating that the operation right of the target page is transferred from the first terminal to the second terminal indicated by the user), and the second terminal detects an operation on the target page on the second terminal after receiving the right transfer instruction.

As shown in fig. 1, the server 103 is connected to the terminal 101 through a network, which may be used to provide services with on-screen requirements (such as game services, teleconferencing services, on-screen explanation services, etc.) for the terminal or a user end installed on the terminal, and the database 105 may be provided on the server or separately from the server, and is used to provide data storage services for the server 103, where the network includes but is not limited to: the terminal 101 is not limited to a PC, a mobile phone, a tablet computer, and the like.

The same-screen interaction method of the embodiment of the present application may be executed by the terminal 101, that is, one of the first terminal or the second terminal, or may be executed by both the first terminal and the second terminal, which is described below as an example. The terminal executing the on-screen interaction method according to the embodiment of the present application may also be executed by a user side installed thereon.

Fig. 2 is a flowchart of an alternative on-screen interaction method according to an embodiment of the present application, and as shown in fig. 2, the method may include the following steps:

step S202, in the process of performing the same-screen interaction between the first terminal and the second terminal, the first terminal detects a first operation, the first operation is used for indicating to transfer the operation right of the target page from the first terminal to the second terminal, and the target page comprises the content of the same-screen interaction.

It can be understood that, a target page displayed by the first terminal and the second terminal (the target page is located in the display interface of the terminal, and if the target page is full screen, the target page and the display interface are considered to be completely overlapped), there is a permission transfer control converted with the page operation right, the permission transfer control may be located in the page, may also be located in a lower part region of the page, may also be located in an upper part region of the page, the control itself may be fixedly displayed, and may also exist in a furled form (the permission transfer control may be displayed through sliding, clicking, double clicking, etc.), and when the permission transfer control is clicked, the background obtains the first operation.

The scheme mainly relates to an operation right transfer function in the same-screen display (and other same-screen applicable scenes are possible), wherein the operation right is the right of an owner of the equipment to operate the equipment, and can be understood as various operations of a user on a target page, and an end user without the operation right cannot operate on the target page. In the process of the same screen, the screen operation behavior can be played back (because the information is transmitted instantly, the 'playback' delay is not sensed by the user), namely, one party (namely, the first terminal or the second terminal) operates the same page, and the other party (namely, the second terminal or the first terminal) can check the click behavior in the operation of the one party in real time to display the same screen; the same screen also supports the synchronization of the change on the screen to another screen for playing, wherein the change is the information synchronization in the non-operation or filling process, such as sliding, clicking and the like; and also filling operation, such as filling user information.

Further, the second terminal may be any one of a plurality of clients, taking an application to an internet interpreter as an example, where the interpreter is used as the first terminal and corresponds to one or more clients, when the first terminal needs to transfer the operation right to one of the plurality of clients, the first operation of the target page is triggered, the background detects the first operation, and at this time, the one of the plurality of clients is used as the second terminal. Furthermore, when the target page triggers the first operation, the user side can be selected, so that the operation right is transferred to the selected user side as the second terminal.

Step S204, in response to the first operation, the first terminal sends an authority transfer instruction to the second terminal to allow the second terminal to operate the target page, so as to realize the exchange of the operation authority between (the user of) the first terminal and (the user of) the second terminal.

The same-screen operation right means that demonstration sharing of operation can be carried out on display contents in own equipment in operation, and the exchange operation right can be understood as transfer of the operation right, namely that the operation right of the user on the current screen can be transferred to another person, so that the other person can operate the screen of the user, and the effect of seeing the user and explaining the user in another operation can be met.

Step S206, the second terminal receives the authority transfer instruction sent by the first terminal.

And step S208, responding to the authority transfer instruction, and detecting the operation performed on the target page on the second terminal.

Further, when the second terminal responds to the permission transfer instruction, the background locks the target page of the first terminal, and unlocks the target page of the second terminal, that is, the target page cannot be operated at the first terminal, but the target page can be operated at the second terminal.

Through the steps, in the process of carrying out the same-screen interaction, the screen operation behavior playback and the screen change of the same page can be synchronized to the screen of the other terminal from one terminal in real time by the first terminal and the second terminal, and the operation right can be transferred to the second terminal where the appointed user is located on the first terminal (and can be recovered according to needs in the operation process and have the capacity of secondary operation).

The same-screen explanation in the related technology is that one player looks over, only one page is operated and reproduced, and the operation right can not be transferred under the real-time display state aiming at the same page.

The technical solution of the present application is further detailed below with reference to specific implementation steps:

step 1: and sending the target page to the second terminal so as to synchronously display the target page on the second terminal, wherein the target page is provided with a plurality of elements which are allowed to be operated, such as a webpage sliding element, a webpage switching element, a text box, a webpage zooming element and the like.

Optionally, if the first terminal needs to explain users of multiple terminals, the target page displayed on the first terminal is sent to the terminal where each user is located.

Optionally, the target page includes one or more web pages, if the target page includes a plurality of web pages, the conversion between the plurality of web pages may be supported, and the operation right of the present application refers to an operation right for all the web pages, in other words, if the second terminal does not need to ask for the operation right again after switching the web pages, the operation flow may be simplified, and the method is more humanized.

And 2, executing an event monitoring task on the first terminal to monitor events occurring on the first terminal, wherein the events mainly comprise two types, one type is an event occurring on a target page of the first terminal, the event needs to be directly synchronized to an interactive opposite terminal (namely a second terminal), and the other type is an operation right change event occurring on the first terminal.

And 3, under the condition that the monitored event is an event occurring on the target page, after the event monitoring task is executed on the first terminal, detecting a second operation on the target page, namely, an operation on an element in the target page on the first terminal, and sending a first event execution instruction to the second terminal (the second operation can be regarded as an event, and the first event execution instruction is equivalent to an instruction for the second terminal to reproduce the second operation, so that the effect that the first terminal and the second terminal are on the same screen is achieved), so that the second operation is reproduced on the second terminal.

Optionally, detecting, on the first terminal, a second operation (i.e., an interactive operation) on the target page, where the second operation includes at least one of: detecting sliding operation on a target page on a first terminal; detecting a zooming operation on a target page on a first terminal; detecting a click operation on a target page on a first terminal; detecting an input operation in a target page on a first terminal; and detecting the switching operation of the pages in the target page on the first terminal.

And 4, under the condition that the monitored event is an operation right change event (namely, the first operation is detected on the first terminal), stopping performing event monitoring on the target page on the first terminal, namely, the user operates the target page on the first terminal, and the target page cannot be responded (namely, the operation cannot be successful) and cannot be synchronized to the second terminal.

In the above scheme, the change of the operation right may be actively changed by the interpreter on the first terminal, and if it is determined that the opposite side needs to operate the target page on the second terminal in the communication, the interpreter triggers a first operation on the first terminal to give the operation right to the opposite side; the operation right change may also be applied by the second terminal to the interpreter, for example, when the user faces a situation that information needs to be filled, a web page needs to be operated, and the like in the process of listening to the explanation, the user initiates an application of the operation right on the second terminal (for example, clicking a virtual button of "application of operation right") which is sent to the first terminal by the second terminal via the network, and the interpreter may select to approve or reject the application on the second terminal, where "approve the application" is equivalent to a first operation triggered on the first terminal.

Optionally, the minimum interactive unit in the web page of the present application is an element, so the operation right may be an operation right for all elements of the entire target web page, or an operation right for one or more target elements, for example, a user may apply for an operation right for a text control on the second terminal to fill in personal information of the user, and meanwhile, the first terminal still has operation rights for other controls except the target element, and can manipulate and explain the web page while the user fills in information.

By adopting the scheme, when the interpreter utilizes the first terminal to explain a certain service to the N second terminals (namely N users), at some places where the N users are required to fill in related information, an interactive target element can be specified for each second terminal, for example, N rows of forms are provided, each row of forms is equivalent to one element, the interpreter can specify one row of forms for each second terminal on the first terminal, so that each user fills in the related information on the second terminal, and the interpreter can still control a target webpage on the first terminal while the users fill in the related information (for example, how to fill in contents, notes, key information prompt in the webpage and the like), thereby improving the working efficiency.

Alternatively, when the first terminal releases the operation right to one second terminal, the operation right excluding some elements (such as information filled in by other second terminals) may be specified; or in the operation process of the second terminal, the operation right of partial elements is recovered, and the interaction efficiency and safety are improved.

And 5, the first terminal sends a permission transfer instruction to the second terminal so as to allow the second terminal to operate the target page.

And 6, the user can operate on the second terminal, and the operation of the user on the target page on the second terminal is detected.

Optionally, an event monitoring task may be executed on the second terminal, so as to perform event monitoring on the target page and perform event monitoring on the operation right change on the second terminal; in a case where the event listening task listens to a fourth operation (similar to the second operation) performed by the element in the target page, a second event execution instruction is sent to the first terminal to reproduce the fourth operation on the first terminal.

And 7, recovering the operation right of the target page at the first terminal, and then restarting the event monitoring on the target page on the first terminal (namely, the operation of the user on the target page on the first terminal can obtain a response, such as inputting characters), thereby synchronizing the interactive operation on the first terminal to the second terminal.

Optionally, a permission recovery control (the specific position and form are similar to those of the permission transfer control and can be set as required) can be displayed in the display interface of the first terminal; receiving a third operation on the permission recovery control, wherein the third operation is used for indicating that the operation right of the target page is recovered from the second terminal to the first terminal, and specifically can be clicking, sliding, pressing and other operations; and receiving the operation right of the target page from the second terminal to the first terminal, and displaying the target page operated by the first terminal to the second terminal in the same screen.

Optionally, similar to the above transferring of the operation right from the first terminal to the second terminal, there are two ways, one of which is that the interpreter actively withdraws on the first terminal, if it is determined that the operation of the opposite party on the target page is completed, a third operation is triggered on the first terminal to instruct to withdraw the operation right of the target page from the second terminal to the first terminal, and a permission withdrawal instruction is sent to the second terminal to stop operating the target page on the second terminal; the operation right change can also be actively released by the second terminal, and the permission recovery notification can be sent out after the second terminal finishes the operation on the target page, at this time, the permission recovery notification is received on the first terminal, and then the operation right of the target page is recovered from the second terminal to the first terminal.

Optionally, on the second terminal, after the first terminal withdraws the operation right of the target page, the event monitoring task is stopped executing on the second terminal.

In the technical scheme of this application, relate to a technique of explanation scheme with screen, through this procedure, explanation person A and explained person B can be same operation page information of synchronization under different screens, explanation person A's operation right can transition to by explanation person B simultaneously, this moment by explanation person B can operate, former explanation person A looks over, if former explanation person A needs to continue the operation, then withdraw the operation right, A can continue the operation this moment, by explanation person B can continue to look over, if there is the needs again, the operation right can be shifted for B by explanation person A at any time, satisfy the comparatively complicated online operation of explanation person and guide. Similarly, the interpreter a can give joint interpretation to a plurality of users such as the interpreter B, C, D, and the operation right can be assigned to transfer, and the operation flow is the same as above.

In an example, as shown in fig. 3, the usage scenario of the exemplary embodiment is that the advisor introduces a product plan and related operations to the user, the overall operation flow is that the advisor initiates a room with on-screen explanation to the user, and the user receives a link of a applet card and needs to agree with the related protocol after clicking; after the protocol is confirmed, the consultant starts to enter the on-screen explanation, the user can only check at this time, when the user is required to fill in information or the user explains that the user needs to check, the consultant hands over the operation right to the user at this time, the user performs the operation or the filling, the user performs the operation, when the user finishes the operation and needs the consultant to explain or confirm again, the consultant can withdraw the operation right, and at this time, the consultant operates and the user watches. The operation and the watching are separated and can be mutually transferred, in the internet sales configuration explanation process, the vision, the hearing and the touch are fully synchronized, the user understanding is facilitated, and the efficiency is obviously improved compared with the prior telephone explanation person and the prior character explanation person.

In addition, when a plurality of people enter the room, the counselor can transfer the operation right to different users respectively, the user receiving the operation right can operate, other users and the counselor watch at the moment, and the counselor operates and other users watch at the moment when the counselor withdraws the operation right.

The operation flow on the advisor side is as follows:

step S1: the background system sends the on-screen explanation invitation to the user and creates a room with a corresponding validity time within which any user can enter the room.

Step S2: a link is generated based on the room and presented in card form at the front end, the advisor clicks on the card, and the system pops up an applet window.

After the user enters the line, the process proceeds to step S3, the user enters the line, the consultant informs the user about the associated authority, and the explanation starts by clicking a button, as shown in fig. 3.

Step S4: after the line is fed, the first frame synchronization of the page is detected, the synchronization is completed, if the page is not synchronized, the page needs to be synchronized first, as shown in fig. 4, and if the page is synchronized, as shown in fig. 5, the page content can be explained on the same screen. In the same-screen explanation process, besides displaying page contents, related controls such as a transfer control of an operation right, "operation transfer", a recovery control of the operation right, "operation recovery", a control of a microphone and the like are also displayed; the position of the related permission transfer control is not specifically limited, and the related permission transfer control may be in a folding display mode, a floating frame display mode, or a toolbar display mode (as illustrated in fig. 6).

Step S5: when the operation right needs to be handed over to the user, the user clicks the bottom "operation hand over", and after confirmation, the user performs the operation and the consultant checks it, as shown in fig. 6.

Step S6: when the operation by the counselor is required during the explanation, the operation retraction button is clicked, and the retraction operation right is confirmed as shown in fig. 7 and 8, and the explanation is continued.

Step S7: after the explanation is finished, the end button (the button at the lower right corner as shown in fig. 8) is clicked, and the explanation is exited after the confirmation is finished.

The viewing and operating process corresponding to the user side is as follows:

step S1: the user receives the invitation and clicks on the card entry as shown in figure 9.

Step S2: after the user enters the line, the consultant explains the same-screen protocol, and the user needs to click the bottom button and enter the next step after agreeing.

Step S3: after incoming line, the first frame synchronization of the page is detected, the synchronization is completed, the page is synchronized firstly if the page is not synchronized, the explanation of the advisor can be listened after the synchronization, and the conversation can be carried out.

Step S4: in the explanation process, the user needs to perform an operation, at this time, after the consultant transfers the operation right, the user side receives a reminder, and as shown in fig. 10, the user can perform an autonomous operation after clicking the ok button to approve.

Step S5: when the operation is completed, the consultant wants to perform the operation, and at this time, the consultant withdraws the operation right and the user side receives the reminder.

Step S6: the explanation is over, at which point the advisor exits and the user can do his or her own work on the current page.

As shown in fig. 6 and 10, the main operations are concentrated on the bottom of the page, the left side is the presentation of the user information avatar, the right side is the function operation button, and the advisor side functions are: operation right transfer/recovery, microphone switch and hang-up function; the user side functions are as follows: microphone switch, hang-up function, fig. 6 and 10 are schematic diagrams showing that the specific position and shape of each control can be customized and configured according to the requirement.

In the implementation scheme based on page on-screen explanation and exchangeable operation rights, the system technically comprises two cores, namely instant messaging and event restoration. The operation of the consultant or the user is transmitted to the user on the other side in real time, the user receives the operation and distributes the operation to the event bound by the page, and the defined event types are as follows: the method comprises the steps of first connection, disconnection reconnection, on-screen request, mutual operation permission conversion, page clicking, page sliding and the like, and the steps are gradually restored to the user on the other side.

As shown in FIG. 11, the detailed steps in conjunction with the operation of the consultant or user are as follows:

in step S1, when the counselor or user enters the page, it sends instruction to inform the user on the other side to enter, when the user on the other side enters, the page prompt changes to prompt both sides to access, at this time, both sides will keep a long connection state.

Step S2, the consultant first sends the same screen invitation to the user, waits for the user to receive, the user can choose to reject, after rejecting, the double-sending connection is disconnected, and the same screen is finished. After agreement, the double-sender will automatically turn on the voice and synchronize the advisor's current page information to the user, starting on the same screen.

In step S3, the consultant starts to perform the on-screen interpretation, and the page viewed by the user side is a relatively static page and cannot be operated. The consultant may click, slide, enter text, etc. on the page, at which time the action on a particular element is calculated in real time. For example: when the consultant slides up and down frequently on the sliding page, but the data cannot be transmitted to the other side in real time, because the consultant side may not complete the sliding, the data sliding in real time is temporarily stored in the local of the user, and after the sliding is finished, the data is completely transmitted to the other side, so that the current data can be ensured to be completed by one operation.

Step S4, as shown in FIG. 12, in order to find the position of ". child-9", the page component can be regarded as a tree structure, and the unique position of the element is calculated from bottom to top by using a depth recursion mode, such as:

and after the element position is obtained, the action to be executed by the element and the corresponding operation value are sent to the user on the other side through an instruction.

And step S5, after receiving the command, the user restores the action types one by one according to the planned action types to ensure that the action types are consistent with the page of the advisor side.

In step S6, either the consultant or the user can actively close the same screen, and after the same screen is finished, the page synchronization state can be released, so that both the consultant and the user can operate the page without synchronizing to the other side.

If abnormal scenes such as network, incoming call, switching to other APP and the like exist, the user at the other side is informed in real time, the user is informed of the disconnection, a session reconnection mechanism is reserved, after the user is reconnected, the pages are synchronized again, and the states of the pages at the two sides are kept consistent.

In the technical scheme of the application: the uniqueness calculation of element positions can be carried out, a page is converted into a ROOT tree in advance, the position of each element under the whole tree is calculated by adopting a recursion mode according to a certain sequence (such as bottom-up, bottom-up and the like) of all elements in the page, the DOM can be endowed to the minimum element for the subsequent page operation behavior, when interaction is carried out between a first terminal and a second terminal, the element positions and corresponding operation are directly transmitted, the element content is not transmitted, the requirement on network bandwidth is reduced, the same-screen performance is improved, and when the operation of the element of 'operation handover' is detected, the first terminal can directly inform the second terminal of the operation of the element position of the element of 'operation handover' and the operation of 'click'; 2) the event-driven pages are consistent with one screen, an event-driven mode is adopted, behavior and actions made by a user are recorded and are transmitted to the user on the other side of the same screen in real time through a webSocket, the actions to be executed are analyzed by utilizing transmitted data, and the events are restored on the pages one by one; 3) the technical implementation of the exchange of the operation right is that after the operation right is handed over to the user by the advisor, the advisor can not operate the page any more, at this time, the page is locked, the event monitoring of the whole page can be unloaded after the locking, the page sliding is prohibited, and at this time, the page at the user side is ensured not to be on the same screen.

The application relates to a method for helping a consultant to share a page under a mobile phone screen to the user by voice under the condition that the user does not know detailed parameters of a scheme book, so that the user can effectively and attentively understand the explanation of the consultant. The user can make the consultant convert the operation right to the user in an unknown place, and the user marks the operation right, so that the two parties can directly and effectively communicate with each other, help the user master the scheme book configured by the user, and select a more appropriate guarantee.

The technical side establishes the authority control of real-time voice call and sharing on the same screen, on one hand, the consultant can solve the confusion for the user from the perspective of insurance profession, and compare different scheme books to help the user to make a selection, thereby avoiding the situation that the user has difficulty in selecting, on the other hand, the application range of the technology is expanded, and more flow and income are brought to other business scenes of a company.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.

Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.

According to another aspect of the embodiment of the application, an on-screen interaction device for implementing the on-screen interaction method is also provided. Fig. 13 is a schematic diagram of an alternative on-screen interaction device according to an embodiment of the present application, and as shown in fig. 13, the device may include:

a first detecting unit 131, configured to detect a first operation on a first terminal, where the first operation is used to instruct to transfer an operation right of a target page from the first terminal to a second terminal, the first terminal and the second terminal are terminals performing same-screen interaction, and the target page includes content of the same-screen interaction;

a sending unit 133, configured to send, in response to the first operation, a permission transfer instruction to the second terminal, so as to allow the second terminal to operate on the target page.

Optionally, the first detection unit is further configured to: before a first operation is detected on a first terminal, executing an event monitoring task on the first terminal, wherein the event monitoring task is used for performing event monitoring on the target page and performing event monitoring on operation right change on the first terminal; and after detecting the first operation on the first terminal, stopping performing event monitoring on the target page on the first terminal.

Optionally, the sending unit is further configured to: before executing the event listening task on the first terminal, the method further includes: sending the target page to the second terminal so as to synchronously show the target page on the second terminal, wherein the target page is provided with a plurality of elements allowed to be operated; the first detection unit is further configured to: after the event monitoring task is executed on the first terminal, detecting a second operation on the target page on the first terminal, wherein the second operation is an operation on an element in the target page; and sending a first event execution instruction to the second terminal so as to reproduce the second operation on the second terminal.

Optionally, the first detection unit is further configured to perform at least one of: detecting sliding operation on the target page on the first terminal; detecting a zooming operation on the target page on the first terminal; detecting a click operation on the target page on the first terminal; detecting an input operation in the target page on the first terminal; and detecting the switching operation of the pages in the target page on the first terminal.

Optionally, the first detection unit is further configured to: after stopping monitoring the event on the target page on the first terminal, and after the first terminal recovers the operation right of the target page, restarting monitoring the event on the target page on the first terminal.

Optionally, the first detection unit is further configured to: detecting the first operation on the first terminal under the condition that the target page needs to be operated on the second terminal; or, after receiving the application of the operation right of the second terminal, the first operation indicating the consent application is detected on the first terminal.

Optionally, the first detecting unit is further configured to reclaim the operation right of the target page as follows: detecting a third operation on the first terminal, wherein the third operation is used for indicating that the operation right of the target page is recovered from the second terminal to the first terminal; responding to the third operation, sending a permission recovery instruction to the second terminal so as to stop operating the target page on the second terminal; or receiving a permission retraction notification on the first terminal, wherein the permission retraction notification is sent out after the operation on the target page is finished on the second terminal; and in response to the permission retraction notification, retracting the operation right of the target page from the second terminal to the first terminal.

Optionally, the first detecting unit 131 is further configured to: displaying a target page in a display interface of a first terminal, wherein the display interface comprises a permission transfer control; receiving a first operation on the permission transfer control, wherein the first operation is used for indicating that the operation right of the target page is transferred from the first terminal to a second terminal, the first terminal and the second terminal are terminals for performing same-screen interaction, and the target page comprises same-screen interaction content; a sending unit 133, configured to respond to the first operation, where the first terminal displays a target page operated by the second terminal.

Optionally, the first detecting unit 131 is further configured to: after the first terminal shows a target page operated by the second terminal, the method further comprises: displaying an authority recovery control in a display interface of the first terminal; receiving a third operation on the permission retraction control, wherein the third operation is used for indicating that the operation right of the target page is retracted from the second terminal to the first terminal; and responding to the first operation, withdrawing the operation right of the target page from the second terminal to the first terminal, and displaying the target page operated by the first terminal to the second terminal in the same screen.

Optionally, the first detecting unit 131 is further configured to: before receiving the first operation on the permission transfer control or after the operation right of the target page is recovered from the second terminal to the first terminal, the method further comprises the following steps: detecting interactive operation between the first terminal and the target page; synchronizing the interactive operation to the second terminal to reproduce the interactive operation on the second terminal.

According to another aspect of the embodiment of the application, an on-screen interaction device for implementing the on-screen interaction method is also provided. Fig. 14 is a schematic diagram of an alternative on-screen interaction device according to an embodiment of the present application, and as shown in fig. 14, the device may include:

a receiving unit 141, configured to receive a permission transfer instruction sent by a first terminal, where the permission transfer instruction is used for the first terminal to instruct to transfer an operation right of a target page from the first terminal to a second terminal, where the first terminal and the second terminal are terminals performing same-screen interaction, and the target page includes content of the same-screen interaction;

a second detecting unit 143, configured to detect, on the second terminal, an operation performed on the target page in response to the permission transfer instruction.

Optionally, the apparatus of the present application may further comprise a processing unit configured to: after receiving a permission transfer instruction sent by a first terminal, executing an event monitoring task on a second terminal, wherein the event monitoring task is used for performing event monitoring on the target page and performing event monitoring on operation right change on the second terminal; the second detection unit is further configured to send a second event execution instruction to the first terminal to reproduce a fourth operation performed on an element in the target page when the event listening task monitors the fourth operation performed on the element in the target page, where the second event execution instruction carries an element position of an operated element.

Optionally, the processing unit is further configured to: after the second terminal detects the operation on the target page, receiving an authority disabling instruction sent by the first terminal, wherein the authority disabling instruction is used for indicating that the operation right of the target page is recovered from the second terminal to the first terminal; and stopping executing the event monitoring task on the second terminal.

Through the steps, in the process of carrying out same-screen interaction, the screen operation behavior playback and the screen change of the same page can be synchronized to the screen of the other terminal from one terminal in real time by the first terminal and the second terminal, and the operation right can be transferred to the second terminal where the appointed user is located on the first terminal (and can be recovered according to needs in the operation process and have the capacity of secondary operation).

It should be noted here that the modules described above are the same as the examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the above embodiments. It should be noted that the modules described above as a part of the apparatus may be operated in a hardware environment as shown in fig. 1, and may be implemented by software, or may be implemented by hardware, where the hardware environment includes a network environment.

According to another aspect of the embodiment of the application, a server or a terminal for implementing the same-screen interaction method is also provided.

Fig. 15 is a block diagram of a terminal according to an embodiment of the present application, and as shown in fig. 15, the terminal may include: one or more processors 1501 (only one of which is shown), a memory 1503, and a transmission device 1505, as shown in fig. 15, the terminal may further include an input output device 1507.

The memory 1503 may be used to store software programs and modules, such as program instructions/modules corresponding to the method and apparatus for interacting with the same screen in the embodiment of the present application, and the processor 1501 executes various functional applications and data processing by running the software programs and modules stored in the memory 1503, so as to implement the method for interacting with the same screen. The memory 1503 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 1503 may further include memory located remotely from processor 1501, which may be connected to a terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 1505 is used for receiving or transmitting data via a network, and may also be used for data transmission between a processor and a memory. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 1505 includes a Network adapter (NIC) which can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmission device 1505 is a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.

Wherein, in particular, the memory 1503 is used for storing application programs.

Processor 1501 may call an application stored in memory 1503 via transfer device 1505 to perform the following steps:

detecting a first operation on a first terminal, wherein the first operation is used for indicating that the operation right of a target page is transferred from the first terminal to a second terminal, the first terminal and the second terminal are terminals for performing same-screen interaction, and the target page comprises content of same-screen interaction;

and responding to the first operation, and sending a permission transfer instruction to the second terminal so as to allow the second terminal to operate the target page.

Processor 1501 is further configured to perform the following steps:

receiving a permission transfer instruction sent by a first terminal, wherein the permission transfer instruction is used for the first terminal to instruct to transfer the operation right of a target page from the first terminal to a second terminal, the first terminal and the second terminal are terminals for performing same-screen interaction, and the target page comprises same-screen interaction content;

and responding to the authority transfer instruction, and detecting the operation on the target page on the second terminal.

By adopting the method and the device, in the process of carrying out same-screen interaction, the screen operation behavior playback of the same page and the change on the screen can be synchronized to the screen of the other terminal from one terminal in real time, and the operation right can be transferred to the second terminal where the appointed user is located on the first terminal (and can be recovered according to needs in the operation process and have the capacity of secondary operation).

Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment is not described herein again.

It can be understood by those skilled in the art that the structure shown in fig. 15 is only an illustration, and the terminal may be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, and a Mobile Internet Device (MID), a PAD, etc. Fig. 15 is a diagram illustrating a structure of the electronic device. For example, the terminal may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 15, or have a different configuration than shown in FIG. 15.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

Embodiments of the present application also provide a storage medium. Alternatively, in this embodiment, the storage medium may be a program code for executing the on-screen interaction method.

Optionally, in this embodiment, the storage medium may be located on at least one of a plurality of network devices in a network shown in the above embodiment.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

Optionally, the storage medium is further arranged to store program code for performing the steps of:

Optionally, in this embodiment, the storage medium may include, but is not limited to: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

The integrated unit in the above embodiments, if implemented in the form of a software functional unit and used as a separate product interpreter or a user, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including instructions for causing one or more computer devices (which may be personal computers, servers, network devices, or the like) to execute all or part of the steps of the method described in the embodiments of the present application.

In the above embodiments of the present application, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In some embodiments provided in the present application, it should be understood that the disclosed ue can be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A method for interacting on the same screen is characterized by comprising the following steps:

2. The method of claim 1,

before detecting the first operation on the first terminal, the method further comprises: executing an event monitoring task on the first terminal, wherein the event monitoring task is used for performing event monitoring on the target page and performing event monitoring on the operation right change on the first terminal;

after detecting the first operation on the first terminal, the method further comprises: and stopping monitoring the event of the target page on the first terminal.

3. The method of claim 2,

before executing the event listening task on the first terminal, the method further includes: sending the target page to the second terminal so as to synchronously show the target page on the second terminal, wherein the target page is provided with a plurality of elements allowed to be operated;

after the event listening task is executed on the first terminal, the method further includes: detecting a second operation on the target page on the first terminal, wherein the second operation is an operation on an element in the target page; and sending a first event execution instruction to the second terminal so as to reproduce the second operation on the second terminal.

4. The method of claim 3, wherein detecting the second operation on the target page on the first terminal comprises at least one of:

detecting sliding operation on the target page on the first terminal;

detecting a zooming operation on the target page on the first terminal;

detecting a click operation on the target page on the first terminal;

detecting an input operation in the target page on the first terminal;

and detecting the switching operation of the pages in the target page on the first terminal.

5. The method of claim 2, wherein after stopping the event listening to the target page on the first terminal, the method further comprises:

and after the first terminal recovers the operation right of the target page, restarting the event monitoring of the target page on the first terminal.

6. The method of claim 1, wherein detecting the first operation at the first terminal comprises:

under the condition that the target page needs to be operated on the second terminal, detecting the first operation triggered by a user on the first terminal; or the like, or, alternatively,

after receiving an operation right application of the second terminal, the first operation indicating an approval application is detected on the first terminal.

7. The method according to any one of claims 1 to 6, further comprising reclaiming the operation right of the target page as follows:

detecting a third operation on the first terminal, wherein the third operation is used for indicating that the operation right of the target page is recovered from the second terminal to the first terminal; responding to the third operation, sending a permission recovery instruction to the second terminal so as to stop operating the target page on the second terminal; or the like, or, alternatively,

receiving a permission retraction notification on the first terminal, wherein the permission retraction notification is sent out after the second terminal finishes the operation on the target page; and in response to the permission retraction notification, retracting the operation right of the target page from the second terminal to the first terminal.

8. A method for interacting on the same screen is characterized by comprising the following steps:

9. The method of claim 8,

after receiving the permission transfer instruction sent by the first terminal, the method further comprises the following steps: executing an event monitoring task on the second terminal, wherein the event monitoring task is used for performing event monitoring on the target page and performing event monitoring on the operation right change on the second terminal;

detecting, at the second terminal, an operation performed on the target page, including: and sending a second event execution instruction to the first terminal to reproduce a fourth operation on the first terminal under the condition that the event monitoring task monitors the fourth operation executed by the element in the target page, wherein the second event execution instruction carries the element position of the operated element.

10. The method according to claim 9, wherein after the second terminal detects the operation performed on the target page, the method further comprises:

receiving a permission deactivation instruction sent by the first terminal, wherein the permission deactivation instruction is used for indicating that the operation right of the target page is recovered from the second terminal to the first terminal;

and stopping executing the event monitoring task on the second terminal.

11. A method for interacting on a screen, the method comprising:

displaying a target page in a display interface of a first terminal, wherein the display interface comprises a permission transfer control;

receiving a first operation on the permission transfer control, wherein the first operation is used for indicating that the operation right of the target page is transferred from the first terminal to a second terminal, the first terminal and the second terminal are terminals for performing same-screen interaction, and the target page comprises same-screen interaction content;

and responding to the first operation, and displaying a target page operated by the second terminal by the first terminal.

12. The method of claim 11, wherein after the first terminal presents the target page operated by the second terminal, the method further comprises:

displaying an authority recovery control in a display interface of the first terminal;

receiving a third operation on the permission retraction control, wherein the third operation is used for indicating that the operation right of the target page is retracted from the second terminal to the first terminal;

and responding to the first operation, withdrawing the operation right of the target page from the second terminal to the first terminal, and displaying the target page operated by the first terminal to the second terminal in the same screen.

13. The method of claim 11, wherein before receiving the first operation on the permission transfer control or after reclaiming the operation right of the target page from the second terminal to the first terminal, the method further comprises:

detecting interactive operation between the first terminal and the target page;

synchronizing the interactive operation to the second terminal to reproduce the interactive operation on the second terminal.

14. The utility model provides a same screen interaction system which characterized in that, includes first terminal and the second terminal that carries out same screen interaction, wherein:

the first terminal is configured to: sending a permission transfer instruction to the second terminal under the condition of detecting a first operation, wherein the first operation is used for indicating that the operation permission of a target page is transferred from the first terminal to the second terminal, the permission transfer instruction is used for indicating that the target page is allowed to be operated on the second terminal, and the target page comprises content interacted with the same screen;

the second terminal is configured to: and after receiving the permission transfer instruction sent by the second terminal, detecting the operation performed on the target page on the second terminal.

15. An on-screen interaction device, comprising:

the first detection unit is used for detecting a first operation on a first terminal, wherein the first operation is used for indicating that the operation right of a target page is transferred from the first terminal to a second terminal, the first terminal and the second terminal are terminals for performing same-screen interaction, and the target page comprises same-screen interaction content;

and the sending unit is used for responding to the first operation and sending an authority transfer instruction to the second terminal so as to allow the second terminal to operate the target page.

16. An on-screen interaction device, comprising:

a receiving unit, configured to receive a permission transfer instruction sent by a first terminal, where the permission transfer instruction is used for the first terminal to instruct to transfer an operation right of a target page from the first terminal to a second terminal, the first terminal and the second terminal are terminals performing same-screen interaction, and the target page includes content of the same-screen interaction;

and the second detection unit is used for responding to the authority transfer instruction and detecting the operation performed on the target page on the second terminal.

17. A storage medium, characterized in that the storage medium comprises a stored program, wherein the program when executed performs the method of any of the preceding claims 1 to 13.

18. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the method of any of claims 1 to 13 by means of the computer program.