CN114979740A - Terminal interaction method, system, electronic equipment and computer readable storage medium - Google Patents

Terminal interaction method, system, electronic equipment and computer readable storage medium Download PDF

Info

Publication number
CN114979740A
CN114979740A CN202210706846.1A CN202210706846A CN114979740A CN 114979740 A CN114979740 A CN 114979740A CN 202210706846 A CN202210706846 A CN 202210706846A CN 114979740 A CN114979740 A CN 114979740A
Authority
CN
China
Prior art keywords
terminal
control terminal
multimedia data
display screen
trigger
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210706846.1A
Other languages
Chinese (zh)
Inventor
李纪新
郑健
张丽娜
付婧杰
武燕燕
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing QIYI Century Science and Technology Co Ltd
Original Assignee
Beijing QIYI Century Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing QIYI Century Science and Technology Co Ltd filed Critical Beijing QIYI Century Science and Technology Co Ltd
Priority to CN202210706846.1A priority Critical patent/CN114979740A/en
Publication of CN114979740A publication Critical patent/CN114979740A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43078—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen for seamlessly watching content streams when changing device, e.g. when watching the same program sequentially on a TV and then on a tablet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/433—Content storage operation, e.g. storage operation in response to a pause request, caching operations
    • H04N21/4334—Recording operations

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The application discloses a terminal interaction method, a system, electronic equipment and a computer readable storage medium, wherein the method comprises the following steps: sending the multimedia data to a control terminal; receiving an operation instruction fed back by the control terminal, wherein the operation instruction is generated by the control terminal according to the triggering operation aiming at the multimedia data, determining the triggering position and the operation information of the triggering operation in the display screen of the control terminal according to the operation instruction, determining the display position of the triggering operation in the display screen of the controlled terminal according to the triggering position, and outputting the operation information at the display position. By applying the technical scheme provided by the application, the multimedia data of the controlled terminal is synchronously sent to the control terminal, the control operation of the control terminal for the multimedia data is synchronously sent to the controlled terminal, and then the double-screen interaction is realized, so that the operation mode is more convenient and faster, more diversified information interaction can be carried out, and the user experience is effectively improved.

Description

Terminal interaction method, system, electronic equipment and computer readable storage medium
Technical Field
The present application relates to the field of multimedia technologies, and in particular, to a terminal interaction method, system, electronic device, and computer-readable storage medium.
Background
With the rapid development of science and technology, multimedia devices such as smart televisions are gradually popularized, and programs such as network movies and television series can be watched through a wired network or a wireless network. However, in an interactive mode, the smart television is not obviously different from a traditional television, the content displayed by the equipment is controlled by a remote controller or an application similar to the remote controller on the mobile phone APP, the information interaction can not be directly realized by touching a screen like watching a video by using a mobile phone, the operation mode is not convenient and fast, and the operation content is limited.
Therefore, how to perform more convenient, faster and diversified information interaction with the multimedia device is a problem to be urgently solved by those skilled in the art.
Disclosure of Invention
In view of the above, to solve the above technical problem, the present application provides a terminal interaction method, a system, an electronic device and a computer-readable storage medium.
In a first aspect, the present application provides a terminal interaction method applied to a controlled terminal, including:
sending the multimedia data to a control terminal;
receiving an operation instruction fed back by the control terminal, wherein the operation instruction is generated by the control terminal according to the triggering operation aiming at the multimedia data;
determining a trigger position and operation information of the trigger operation in a display screen of the control terminal according to the operation instruction;
and determining the display position of the trigger operation in the display screen of the controlled terminal according to the trigger position, and outputting the operation information at the display position.
Optionally, a first communication channel and a second communication channel are established between the control terminal and the controlled terminal;
the sending of the multimedia data to the control terminal includes:
sending the multimedia data to the control terminal through the first communication channel;
the receiving of the operation instruction fed back by the control terminal includes:
and receiving the operation instruction fed back by the control terminal through the second communication channel.
Optionally, the sending the multimedia data to the control terminal includes:
recording a screen of the display screen of the controlled terminal to obtain the multimedia data;
and sending the multimedia data to the control terminal.
Optionally, the determining, according to the trigger position, a display position of the trigger operation in a display screen of the controlled terminal includes:
and determining a display position corresponding to the trigger position according to a coordinate mapping relation, wherein the coordinate mapping relation is a mapping relation between a control terminal display screen coordinate system and a controlled terminal display screen coordinate system.
Optionally, the method further comprises:
monitoring the playing progress of the multimedia data in the display screen of the controlled terminal;
and when the playing progress reaches a preset playing node, sending push information related to the multimedia data to the control terminal so that the control terminal outputs the push information to a display screen of the control terminal, and the push information is superposed on the multimedia data for displaying.
Optionally, the method further comprises:
receiving a push request sent by the control terminal;
determining pushing information about the multimedia data according to the pushing request;
and sending the push information to the control terminal so that the control terminal outputs the push information to a display screen of the control terminal, and the push information is superposed on the multimedia data for display.
In a second aspect, the present application provides another terminal interaction method, applied to a control terminal, including:
receiving multimedia data sent by a controlled terminal;
acquiring a trigger operation aiming at the multimedia data;
generating an operation instruction according to the trigger operation, wherein the operation instruction comprises a trigger position and operation information of the trigger operation in a display screen of a control terminal;
and sending the operation instruction to the controlled terminal so that the controlled terminal determines the display position of the trigger operation in the display screen of the controlled terminal according to the trigger position and outputs the operation information at the display position.
In a third aspect, the present application provides a terminal interaction system, comprising a controlled terminal and a control terminal, wherein,
the control terminal is used for receiving the multimedia data sent by the controlled terminal, acquiring trigger operation aiming at the multimedia data, generating an operation instruction according to the trigger operation and sending the operation instruction to the controlled terminal;
the controlled terminal is used for sending the multimedia data to the control terminal, determining a trigger position and operation information of the trigger operation in a display screen of the control terminal according to the operation instruction, determining a display position of the trigger operation in the display screen of the controlled terminal according to the trigger position, and outputting the operation information at the display position.
In a fourth aspect, the present application provides an electronic device, comprising:
a memory for storing a computer program;
and the processor is used for realizing the steps of any terminal interaction method when the computer program is executed.
In a fifth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the terminal interaction methods described above.
According to the terminal interaction scheme, multimedia data are sent to the control terminal; the method comprises the steps of receiving an operation instruction fed back by a control terminal, generating the operation instruction by the control terminal according to triggering operation aiming at multimedia data, determining a triggering position and operation information of the triggering operation in a display screen of the control terminal according to the operation instruction, determining a display position of the triggering operation in the display screen of the controlled terminal according to the triggering position, and outputting the operation information at the display position, so that the multimedia data of the controlled terminal are synchronized to the control terminal, the control operation aiming at the multimedia data of the control terminal is synchronized to the controlled terminal, and further double-screen interaction is realized.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the drawings that are needed to be used in the description of the prior art and the embodiments of the present application will be briefly described below. Of course, the following description of the drawings related to the embodiments of the present application is only a part of the embodiments of the present application, and it will be obvious to those skilled in the art that other drawings can be obtained from the provided drawings without any creative effort, and the obtained other drawings also belong to the protection scope of the present application.
Fig. 1 is a schematic structural diagram of a terminal interaction system according to an embodiment of the present disclosure;
fig. 2 is a schematic flowchart of a terminal interaction method applied to a controlled terminal according to an embodiment of the present disclosure;
fig. 3 is a schematic flowchart of another terminal interaction method applied to a controlled terminal according to an embodiment of the present disclosure;
fig. 4 is a schematic flowchart of another terminal interaction method applied to a controlled terminal according to an embodiment of the present application;
fig. 5 is a schematic flowchart of a further terminal interaction method applied to a controlled terminal according to an embodiment of the present application;
fig. 6 is a schematic flowchart of a terminal interaction method applied to a control terminal according to an embodiment of the present disclosure;
fig. 7 is an application scene diagram of a terminal interaction method according to an embodiment of the present application;
fig. 8 is an application scenario diagram of another terminal interaction method according to the embodiment of the present application;
fig. 9 is a schematic structural diagram of a terminal interaction device according to an embodiment of the present disclosure;
fig. 10 is a schematic structural diagram of another terminal interaction device according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a terminal interaction system provided in an embodiment of the present application, where the terminal interaction system may include a control terminal 100 and a controlled terminal 200, and is used to implement information interaction between the control terminal 100 and the controlled terminal 200, where the information interaction mainly includes synchronizing multimedia data of the controlled terminal 200 to the control terminal 100, and synchronizing control operation of the control terminal 100 on the multimedia data to the controlled terminal 200, so as to implement dual-screen interaction.
The controlled terminal 100 may be a terminal device such as a smart television that can be used to play multimedia data, and the control terminal 200 may be a terminal device such as a mobile phone and a tablet that can perform a touch screen operation.
It is to be understood that the number of the control terminals 100 is not unique, and when the number is plural, multi-terminal interaction may be implemented, for example, a chat room for multimedia data in the controlled terminal 200 may be formed, so as to perform topic discussion, interaction, voting, etc. for the multimedia data.
The embodiment of the application provides a terminal interaction method.
Referring to fig. 2, fig. 2 is a schematic flowchart of a terminal interaction method applied to a controlled terminal according to an embodiment of the present application, where the terminal interaction method includes the following steps S101 to S104:
s101: and sending the multimedia data to the control terminal.
This step is intended to enable the transmission of multimedia data, i.e. the controlled terminal transmits the multimedia data to the control terminal. Specifically, when a communication request of the control terminal is received, the communication request is used for instructing the control terminal to interact with the controlled terminal so as to obtain the multimedia data. In this embodiment, the multimedia data includes, but is not limited to, video, images, and the like.
Specifically, after the controlled terminal obtains the multimedia data, the controlled terminal may first perform compression processing on the multimedia data to obtain compressed multimedia data, that is, the compressed data, and then send the compressed data to the control terminal for subsequent processing.
Correspondingly, after receiving the compressed data sent by the controlled terminal, the control terminal may decompress the compressed data to obtain the multimedia data, and then perform subsequent operations on the multimedia data.
Therefore, the terminal interaction method provided by the embodiment of the application transmits the multimedia data in the form of the compressed packet when the controlled terminal and the control terminal interact with each other, so that the data transmission efficiency can be effectively improved, and the real-time performance of on-screen display is further ensured.
S102: and receiving an operation instruction fed back by the control terminal, wherein the operation instruction is generated by the control terminal according to the triggering operation aiming at the multimedia data.
This step is intended to enable the reception of an operation instruction, which is an operation instruction for multimedia data, generated by the control terminal according to a trigger operation for the multimedia data. Specifically, after receiving the multimedia data sent by the controlled terminal, the control terminal can output the multimedia data to a display screen of the control terminal, so that the multimedia data can be synchronously displayed between the control terminal and the controlled terminal; furthermore, in the display process, the user can perform diversified processing on the multimedia data through touch screen operation, such as praise, comment posting, expression package sending, voting and the like; meanwhile, the control terminal collects the triggering operation of the users for the multimedia data and generates corresponding operation instructions; and finally, feeding back the operation instruction to the controlled terminal.
S103: and determining a trigger position and operation information of the trigger operation in the display screen of the control terminal according to the operation instruction.
This step is intended to implement the parsing of the operation instructions. Specifically, after receiving an operation instruction fed back by the control terminal, the trigger position and the operation information of the trigger operation carried by the control terminal in the display screen of the control terminal can be obtained through instruction analysis. The operation information is specific operations, such as praise, text comments, expression package comments and the like, made by the user for the multimedia data; the trigger position is a specific position (specifically, coordinate information) where the operation data occurs on the display screen of the control terminal, such as a praise position, a display position of comment text, a display position of an emoticon, and the like, and the trigger position is used for determining the specific position of the operation data on the display screen of the controlled terminal.
S104: and determining the display position of the trigger operation in the display screen of the controlled terminal according to the trigger position, and outputting operation information at the display position.
This step is intended to enable the execution of the operating instructions. And after the operation instruction is analyzed, the operation information can be displayed on a display screen of the controlled terminal according to the determined trigger position and the operation information. Therefore, the control terminal and the controlled terminal synchronously display the operation events aiming at the multimedia data.
For example, when the trigger operation of the control terminal is like the like aiming at the multimedia data, the display screen of the controlled terminal synchronously displays the like animation special effect; when the triggering operation of the control terminal is the comment aiming at the multimedia data, the controlled terminal display screen synchronously displays the comment content; when the triggering operation of the control terminal is the comment of the emoticon aiming at the multimedia data, the display screen of the controlled terminal synchronously displays the same animation special effect of the emoticon.
Therefore, the terminal interaction method provided by the embodiment of the application transmits the multimedia data to the control terminal; the method comprises the steps of receiving an operation instruction fed back by a control terminal, generating the operation instruction by the control terminal according to triggering operation aiming at multimedia data, determining a triggering position and operation information of the triggering operation in a display screen of the control terminal according to the operation instruction, determining a display position of the triggering operation in the display screen of the controlled terminal according to the triggering position, and outputting the operation information at the display position, so that the multimedia data of the controlled terminal are synchronized to the control terminal, the control operation aiming at the multimedia data of the control terminal is synchronized to the controlled terminal, and further double-screen interaction is realized.
Referring to fig. 3, fig. 3 is a schematic flowchart of another terminal interaction method applied to a controlled terminal according to an embodiment of the present disclosure. The method comprises the following steps:
s201: recording a screen of a display screen of the controlled terminal to obtain multimedia data;
in this embodiment, when terminal interaction is required (i.e., when a communication request of the control terminal is received), the control terminal may establish a communication connection with the controlled terminal, then the controlled terminal records a screen of the controlled terminal to obtain multimedia data, and finally sends the multimedia data to the control terminal through the communication connection. In this embodiment, a screen cast technology may be adopted to record the data information displayed by the display screen of the controlled terminal.
S202: and sending the multimedia data to the control terminal through the first communication channel.
S203: and receiving an operation instruction fed back by the control terminal through a second communication channel.
In this embodiment, for the step S202 and the step S203, in order to implement data communication between the control terminal and the controlled terminal, a first communication channel and a second communication channel are established between the control terminal and the controlled terminal, and information interaction is further implemented through the first communication channel and the second communication channel, and a communication channel may be established between the control terminal and the controlled terminal in advance, so that the communication channel may be used to send multimedia data to the control terminal and receive an operation instruction fed back by the control terminal. When the communication channel is established, the control terminal can be implemented according to the address information of the controlled terminal, and the address information can be specifically the IP address of the controlled terminal, so that the establishment of the communication channel between the control terminal and the controlled terminal is implemented according to the address information.
In a possible implementation manner, the address information may include a first port address and a second port address, and the control terminal establishes a communication channel with the controlled terminal according to the address information of the controlled terminal, which may include the following steps:
the control terminal establishes a first communication channel with the controlled terminal according to the first port address so as to send the multimedia data to the controlled terminal by utilizing the first communication channel;
and establishing a second communication channel with the controlled terminal according to the second port address so as to execute the step of receiving the operation instruction fed back by the control terminal by using the second communication channel.
Specifically, a dual communication channel may be established, including the first communication channel and the second communication channel, so that the controlled terminal transmits multimedia data to the control terminal through the first communication channel, and the controlled terminal receives an operation instruction fed back by the control terminal through the second communication channel. The address information of the controlled terminal may include a first port address and a second port address, for example, the first port address may be: the controlled terminal IP + port number of the first port, and the second port address may be: controlled terminal IP + port number of the second port. Therefore, the control terminal can establish a first communication channel with the controlled terminal by using the first port address for realizing the transmission of the multimedia data; the control terminal can establish a second communication channel with the controlled terminal by using the second port address, so as to realize the transmission of the operation instruction.
In addition, the second communication channel can also be used for the controlled terminal to send other data information besides the multimedia data to the control terminal, for example, when the multimedia data is a movie, the controlled terminal can send the movie data to the control terminal through the first communication channel and simultaneously send the information of the movie character to the control terminal through the second communication channel; for another example, when the number of the controlled terminals is multiple and multi-terminal interaction is performed, the triggering operation of other control terminals for the multimedia data can be sent to the control terminal through the second communication channel. Thus, the second communication channel may be a bidirectional communication channel. On the basis, in order to ensure that the multimedia data can be clearly displayed on the display screen of the control terminal, the first communication channel may be used only for transmitting the multimedia data, that is, the first communication channel may be a one-way communication channel.
Therefore, the terminal interaction method provided by the embodiment of the application synchronizes the multimedia data of the controlled terminal to the control terminal, synchronizes the control operation of the control terminal for the multimedia data to the controlled terminal, and transmits the data by establishing the dual communication channel on the basis of further realizing the terminal interaction, so that the real-time performance of the terminal interaction can be effectively ensured, and the problems of data blockage and transmission delay possibly caused by data transmission by using the single communication channel are avoided; in addition, the first communication channel which is independently established for unidirectional transmission is only used for realizing the transmission of the multimedia data, and the definition of the multimedia data displayed on the display screen of the control terminal can be further ensured.
S204: and determining a trigger position and operation information of the trigger operation in the display screen of the control terminal according to the operation instruction.
In this embodiment, the step S204 is the same as the step S103, and details are not repeated herein in this embodiment, which may specifically refer to the above description.
S205: and determining a display position corresponding to the trigger position according to a coordinate system mapping relation, and outputting operation information at the display position, wherein the coordinate system mapping relation is a mapping relation between a control terminal display screen coordinate system and a controlled terminal display screen coordinate system.
Specifically, a mapping relationship between the coordinate system of the display screen of the control terminal and the coordinate system of the display screen of the controlled terminal, that is, the coordinate mapping relationship may be pre-established, so as to determine the display position of the trigger operation on the display screen of the controlled terminal for the multimedia data in the display screen of the control terminal according to the coordinate mapping relationship. In the implementation process, after the operation instruction is obtained, the operation instruction can be analyzed to obtain the trigger position and the operation information which are carried by the operation instruction and are related to the trigger operation; further, determining a display position of the display screen of the controlled terminal corresponding to the trigger position by referring to the coordinate mapping relation, wherein the trigger position is the display position of the operation data on the display screen of the control terminal; and finally, outputting the operation information at the display position of the display screen of the controlled terminal so that the operation information is displayed at the corresponding position of the display screen of the controlled terminal.
Therefore, according to the terminal interaction method provided by the embodiment of the application, the multimedia data of the controlled terminal is synchronized to the control terminal, the control operation of the control terminal for the multimedia data is synchronized to the controlled terminal, and further, on the basis of realizing terminal interaction, the mapping relation between the coordinate system of the display screen of the control terminal and the coordinate system of the display screen of the controlled terminal is pre-established, so that the display screen of the control terminal and the display screen of the controlled terminal can display completely same pictures, the equal scaling of the displayed pictures is realized, and the improvement of user experience is facilitated.
According to the terminal interaction method provided by the embodiment, in the interaction process of the controlled terminal and the control terminal, the same type of periphery related to the multimedia data can be pushed to the control terminal, and the interaction process of the controlled terminal and the control terminal can be understood as the process of synchronously displaying the multimedia data between the control terminal and the controlled terminal. Wherein, the pushing of the homogeneous periphery related to the multimedia data to the control terminal can be realized according to the following mode.
In a possible implementation manner, referring to fig. 4, in a process of interaction between a controlled terminal and a control terminal, the terminal interaction method provided in this embodiment further includes:
s301: and monitoring the playing progress of the multimedia data in the display screen of the controlled terminal.
S302: when the playing progress reaches a preset playing node, the pushing information about the multimedia data is sent to the control terminal, so that the control terminal outputs the pushing information to a display screen of the control terminal, and the pushing information is superposed on the multimedia data to be displayed.
In this embodiment, the push information is the same type of periphery related to the multimedia data, for example, the push information may be a magazine purchasing link, a shirt purchasing link, and the like related to the multimedia data, and the specific form of the push information may be set according to actual needs, which is not specifically limited in this embodiment. The corresponding relation between the preset playing node and the push information can be stored in the controlled terminal aiming at the multimedia data, the playing node corresponding to the monitoring moment can be obtained by monitoring the playing progress of the multimedia data in the display screen of the controlled terminal, and when the playing node is the preset playing node, the push information can be obtained from the preset corresponding relation.
It can be seen that, according to the terminal interaction method provided by the embodiment of the application, the multimedia data of the controlled terminal is synchronized to the control terminal, the control operation of the control terminal for the multimedia data is synchronized to the controlled terminal, and further, on the basis of terminal interaction, the control terminal monitors the playing progress of the multimedia data in the display screen of the controlled terminal, and sends the obtained push information to the control terminal, so that the push information is displayed on the multimedia data displayed on the display screen of the control terminal, a user can conveniently and quickly place an order, a purchase path is shorter, and when the push information is pushed, the playing of the multimedia data of the control terminal is not interrupted, and the user experience is further improved.
In another possible implementation manner, referring to fig. 5, in the process of the interaction between the controlled terminal and the control terminal, the terminal interaction method provided in this embodiment further includes:
s401: and receiving a push request sent by a control terminal.
S402: push information about the multimedia data is determined according to the push request.
S403: and sending the push information to the control terminal so that the control terminal outputs the push information to a display screen of the control terminal and the push information is superposed on the multimedia data for display.
The push information is consistent with the push information in the previous embodiment, and details are not described herein in this embodiment. In this embodiment, in order to prevent that information is frequently pushed to the control terminal and interaction experience of a user is affected, the pushed information does not need to be actively pushed by the controlled terminal, and can be pushed after receiving a related request of the control terminal, for example, the pushed request can be generated when the control terminal detects a trigger operation for a pushed control, and the pushed control is displayed on a display screen of the control terminal for selection by the user.
Therefore, according to the terminal interaction method provided by the embodiment of the application, the multimedia data of the controlled terminal is synchronized to the control terminal, the control operation of the control terminal for the multimedia data is synchronized to the controlled terminal, and further, on the basis of terminal interaction, the obtained push information corresponding to the push request is sent to the control terminal by receiving the push request sent by the control terminal, so that the control terminal displays the push information in a display screen of the control terminal, the user can conveniently and quickly order, the purchase path is shorter, the controlled terminal is not required to actively push the push information, the user experience is improved, and when the push information is pushed again, the playing of the multimedia data of the control terminal is not interrupted, and the user experience is further improved.
Referring to fig. 6, fig. 6 is a flowchart illustrating a terminal interaction method applied to a control terminal according to an embodiment of the present disclosure, where the terminal interaction method includes the following steps S501 to S504:
s501: and receiving the multimedia data sent by the controlled terminal.
This step is intended to implement the reception of multimedia data, which refers to data information displayed on the display screen of the controlled terminal, including but not limited to video, images, etc.
In this embodiment, the manner of acquiring the multimedia data may refer to the step S201, and is not described in detail in this embodiment. The control terminal receives the multimedia data sent by the controlled terminal through the first communication channel, and the establishment of the first communication channel may refer to the above description, which is not described in this embodiment.
S502: and acquiring a trigger operation aiming at the multimedia data.
The step aims at triggering the acquisition of operation, and after receiving the multimedia data sent by the controlled terminal, the multimedia data can be output to a display screen of the control terminal, so that the multimedia data can be synchronously displayed between the control terminal and the controlled terminal; furthermore, in the display process, the user can perform diversified processing on the multimedia data through touch screen operation, such as praise, comment making, expression package sending, voting and the like; meanwhile, the control terminal collects the triggering operations of the users for the multimedia data.
S503: and generating an operation instruction according to the triggering operation, wherein the operation instruction comprises a triggering position and operation information of the triggering operation in a display screen of the control terminal.
The step aims at generating an operation instruction, and after the control terminal collects the trigger operation of the user for the multimedia data, the corresponding operation instruction is generated according to the trigger operation. The operation instruction is used for indicating the controlled terminal to simulate corresponding trigger operation and is displayed on a display screen of the controlled terminal, so that the control terminal and the controlled terminal can synchronously display operation events aiming at multimedia data.
S504: and sending the operation instruction to the controlled terminal so that the controlled terminal determines the display position of the trigger operation in the display screen of the controlled terminal according to the trigger position and outputs operation information at the display position.
This step is intended to implement sending the operation instruction to the controlled terminal, so that the controlled terminal executes the operation instruction. Specifically, after a corresponding operation instruction is generated according to the trigger operation, the operation instruction can be sent to the controlled terminal through the communication connection; for the controlled terminal, after receiving the operation instruction, the controlled terminal can obtain the display position and the operation information through instruction analysis, and simultaneously the controlled terminal outputs the operation information to the display position of the display screen for display. Therefore, the control terminal and the controlled terminal synchronously display the operation events aiming at the multimedia data. In this embodiment, the operation instruction may be sent through the second communication channel, and the establishment of the second communication channel may refer to the above description, which is not described in detail in this embodiment.
Therefore, according to the terminal interaction method provided by the embodiment of the application, the multimedia data sent by the controlled terminal is received, the triggering operation aiming at the multimedia data is obtained, the operation instruction is generated according to the triggering operation and sent to the controlled terminal, so that the controlled terminal determines the display position of the triggering operation in the display screen of the controlled terminal according to the triggering position and outputs the operation information at the display position, the multimedia data of the controlled terminal is synchronized to the control terminal, the control operation aiming at the multimedia data by the control terminal is synchronized to the controlled terminal, the double-screen interaction is further realized, the operation mode is more convenient and faster, more diversified information interaction can be carried out, and the user experience is effectively improved.
The embodiment of the application provides a terminal interaction method for a specific application scene, taking an example of information interaction between a television terminal and a mobile phone terminal, wherein the television terminal is a controlled terminal, and the mobile phone terminal is a control terminal. Referring to fig. 7, fig. 7 is an application scenario diagram of a terminal interaction method according to an embodiment of the present application, and an implementation process of the application scenario diagram may include:
firstly, a desktop application is installed at a television end as a server end, and a corresponding desktop application is installed at a mobile phone end as a client end.
Further, after the television terminal is started, the desktop application as the server terminal creates two TCP (Transmission Control Protocol) connection channels (Socket connection) and binds to two ports of the television terminal; meanwhile, the desktop application of the mobile phone end is started, and when the mobile phone end and the television end are located in the same network (for example, connected with the same WIFI), the mobile phone end and the television end can establish two communication channels through the two ports so as to perform data communication.
Finally, after the communication channels are established, one of the communication channels is a one-way channel and is used for pushing the display picture (namely the multimedia data) of the television end to the mobile phone end for synchronous display; the other communication channel is a bidirectional channel and is used for sending the trigger operation of the mobile phone end aiming at the display picture to the television end for synchronous display and receiving other related data information which is fed back by the television end and is related to the display picture.
The implementation process of data transmission based on the first communication channel is as follows:
1. and establishing a non-blocking service end at the television end through a ServerSocketChannel type of NIO (New Input/Output).
2. The server creates a Socket and listens to the 6000 port, and waits for connection.
3. The mobile phone terminal serving as the client establishes connection with the television terminal through the 'television terminal IP +6000 port', transmits the connection into an answering method, and receives byte stream data from the server terminal through an onStringAvailable () callback function.
4. The server side obtains byte stream data and prepares to output the byte stream data to the client side; the byte stream data is content being displayed on the Screen of the server, and can be obtained based on a Screen cast technology (Screen projection technology), and the implementation process thereof may include:
(1) applying for recording screen permission and creating an mMediaproject object;
(2) establishing a MediaCodec class for compressing and coding multimedia data to realize a screen recording function;
(3) in the loop body, encodedData (encoded data) is continuously obtained and sent out through Socket.
5. The client decodes the received byte stream through MediaCodec, and then displays the decoded data on a surface view.
Thus, the process of transmitting and displaying the multimedia data from the server (television) to the client (mobile phone) is completed.
Secondly, the implementation process of data transmission based on the second communication channel is as follows:
1. and the mobile phone terminal as the client establishes connection with the television terminal through the television terminal IP +6059 port.
2. And after the server establishes connection with the client, assigning the socket instance to the touchSocket.
3. And the client sends data to the server through the touchSocket instance to complete interaction.
The data sent by the client to the server is a character string in a JSON format, and contains coordinate information of trigger operation on a screen of the mobile phone terminal.
4. After receiving the corresponding data, the server analyzes the data according to the JSON format, and then simulates corresponding trigger operations such as clicking, sliding, dragging and the like by using an InputManager class.
Therefore, the process of transmitting the trigger operation aiming at the multimedia data from the client (mobile phone end) to the server (television end) and displaying the trigger operation is completed. Of course, the server may also send related information about the multimedia data to the client through the communication channel to implement bidirectional communication based on the second communication channel.
In addition, the television side is used as a server side and can be connected with a plurality of mobile phone sides, as shown in fig. 8, fig. 8 is an application scene diagram of another terminal interaction method provided in the embodiment of the present application, and a plurality of mobile phone sides are supported to simultaneously operate the same television side, so that multi-terminal interaction is realized.
Therefore, the terminal interaction method provided by the embodiment of the application realizes that the multimedia data of the television end is synchronized to the mobile phone end, and the control operation of the mobile phone end aiming at the multimedia data is synchronized to the television end, so that the double-screen interaction is realized, the operation mode is more convenient and faster, more diversified information interaction can be performed, and the user experience is effectively improved.
The embodiment of the application provides a terminal interaction device.
Referring to fig. 9, fig. 9 is a schematic structural diagram of a terminal interaction device according to an embodiment of the present application, where the terminal interaction device is applied to a controlled terminal, and the terminal interaction device may include:
the first sending module 1 is used for sending the multimedia data to the control terminal;
the first receiving module 2 is used for receiving an operation instruction fed back by the control terminal, and the operation instruction is generated by the control terminal according to the triggering operation aiming at the multimedia data;
the determining module 3 is used for determining a trigger position and operation information of the trigger operation in the display screen of the control terminal according to the operation instruction;
the determining module 3 is further used for determining the display position of the triggering operation in the display screen of the controlled terminal according to the triggering position;
and the output module 4 is used for outputting the operation information at the display position.
Therefore, the terminal interaction device provided by the embodiment of the application transmits the multimedia data to the control terminal; the method comprises the steps of receiving an operation instruction fed back by a control terminal, generating the operation instruction by the control terminal according to triggering operation aiming at multimedia data, determining a triggering position and operation information of the triggering operation in a display screen of the control terminal according to the operation instruction, determining a display position of the triggering operation in the display screen of the controlled terminal according to the triggering position, and outputting the operation information at the display position, so that the multimedia data of the controlled terminal are synchronized to the control terminal, the control operation aiming at the multimedia data of the control terminal is synchronized to the controlled terminal, and further double-screen interaction is realized.
In one embodiment of the application, a first communication channel and a second communication channel are established between a control terminal and a controlled terminal.
The first sending module 1 is further configured to send the multimedia data to the control terminal through the first communication channel.
The first receiving module 2 is further configured to receive an operation instruction fed back by the control terminal through the second communication channel.
In an embodiment of the present application, the first sending module 1 is further configured to record a screen of a display screen of the controlled terminal, so as to obtain multimedia data;
the first receiving module 2 is further configured to send the multimedia data to the control terminal.
In an embodiment of the present application, the determining module 3 is further configured to determine the display position corresponding to the trigger position according to a coordinate mapping relationship, where the coordinate mapping relationship is a mapping relationship between a control terminal display screen coordinate system and a controlled terminal display screen coordinate system.
In an embodiment of the present application, the terminal interaction apparatus may further include:
and the monitoring module is used for monitoring the playing progress of the multimedia data in the display screen of the controlled terminal.
The first sending module 1 is further configured to send, when the playing progress reaches a preset playing node, the push information about the multimedia data to the control terminal, so that the control terminal outputs the push information to a display screen of the control terminal, and the push information is superimposed on the multimedia data for display.
In an embodiment of the present application, the first receiving module 2 is further configured to receive a push request sent by the control terminal.
The determining module 3 is further configured to determine push information about the multimedia data according to the push request.
The first sending module 1 is further configured to send the push information to the control terminal, so that the control terminal outputs the push information to a display screen of the control terminal, and the push information is superimposed on the multimedia data for display.
For the introduction of the apparatus provided in the embodiment of the present application, please refer to the above method embodiment, which is not described herein again.
The embodiment of the application provides another terminal interaction device.
Referring to fig. 10, fig. 10 is a schematic structural diagram of another terminal interaction device provided in an embodiment of the present application, where the terminal interaction device is applied to a control terminal, and the terminal interaction device may include:
and the second receiving module 5 is used for receiving the multimedia data sent by the controlled terminal.
And the acquisition module 6 is used for acquiring the trigger operation aiming at the multimedia data.
And the generating module 7 is used for generating an operation instruction according to the trigger operation, wherein the operation instruction comprises a trigger position and operation information of the trigger operation in the display screen of the control terminal.
And the second sending module 8 is used for sending the operation instruction to the controlled terminal so that the controlled terminal determines the display position of the trigger operation in the display screen of the controlled terminal according to the trigger position and outputs the operation information at the display position.
Therefore, the terminal interaction device provided by the embodiment of the application transmits the multimedia data to the control terminal; the method comprises the steps of receiving an operation instruction fed back by a control terminal, generating the operation instruction by the control terminal according to triggering operation aiming at multimedia data, determining a triggering position and operation information of the triggering operation in a display screen of the control terminal according to the operation instruction, determining a display position of the triggering operation in the display screen of the controlled terminal according to the triggering position, and outputting the operation information at the display position, so that the multimedia data of the controlled terminal are synchronized to the control terminal, the control operation aiming at the multimedia data of the control terminal is synchronized to the controlled terminal, and further double-screen interaction is realized.
For the introduction of the apparatus provided in the embodiment of the present application, please refer to the above method embodiment, which is not described herein again.
The embodiment of the application provides electronic equipment.
Referring to fig. 11, fig. 11 is a schematic structural diagram of an electronic device provided in the present application, where the electronic device may include:
a memory for storing a computer program;
and the processor is used for realizing the steps of any one of the terminal interaction methods when executing the computer program.
As shown in fig. 11, which is a schematic view of a composition structure of an electronic device, the electronic device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all communicate with each other through a communication bus 13.
In the embodiment of the present application, the processor 10 may be a Central Processing Unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic device, etc. The processor 10 may call a program stored in the memory 11, and in particular, the processor 10 may perform operations in the embodiment of the terminal interaction method.
The memory 11 is used for storing one or more programs, the program may include program codes, the program codes include computer operation instructions, in this embodiment, the memory 11 stores at least the program for implementing the following functions:
sending the multimedia data to a control terminal; receiving an operation instruction fed back by the control terminal, wherein the operation instruction is generated by the control terminal according to the triggering operation aiming at the multimedia data, determining the triggering position and the operation information of the triggering operation in the display screen of the control terminal according to the operation instruction, determining the display position of the triggering operation in the display screen of the controlled terminal according to the triggering position, and outputting the operation information at the display position.
Alternatively, at least a program for realizing the following functions is stored:
the method comprises the steps of receiving multimedia data sent by a controlled terminal, obtaining triggering operation aiming at the multimedia data, generating an operation instruction according to the triggering operation, sending the operation instruction to the controlled terminal, enabling the controlled terminal to determine a display position of the triggering operation in a display screen of the controlled terminal according to the triggering position, and outputting operation information at the display position.
In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created during use.
Further, the memory 11 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid state storage device.
The communication interface 12 may be an interface of a communication module for connecting with other devices or systems.
Of course, it should be noted that the structure shown in fig. 11 does not constitute a limitation to the electronic device in the embodiment of the present application, and in practical applications, the electronic device may include more or less components than those shown in fig. 11, or some components may be combined.
The embodiment of the present application provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of any one of the above-mentioned terminal interaction methods can be implemented.
The computer-readable storage medium may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
For introduction of a computer-readable storage medium provided in the embodiments of the present application, please refer to the above method embodiments, which are not described herein again.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The technical solutions provided by the present application are described in detail above. The principles and embodiments of the present application are explained herein using specific examples, which are provided only to help understand the method and the core idea of the present application. It should be noted that, for those skilled in the art, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall into the protection scope of the present application.

Claims (10)

1. A terminal interaction method is applied to a controlled terminal and comprises the following steps:
sending the multimedia data to a control terminal;
receiving an operation instruction fed back by the control terminal, wherein the operation instruction is generated by the control terminal according to the triggering operation aiming at the multimedia data;
determining a trigger position and operation information of the trigger operation in a display screen of the control terminal according to the operation instruction;
and determining the display position of the trigger operation in the display screen of the controlled terminal according to the trigger position, and outputting the operation information at the display position.
2. The method according to claim 1, wherein a first communication channel and a second communication channel are established between the control terminal and the controlled terminal;
the sending of the multimedia data to the control terminal includes:
sending the multimedia data to the control terminal through the first communication channel;
the receiving of the operation instruction fed back by the control terminal includes:
and receiving the operation instruction fed back by the control terminal through the second communication channel.
3. The method of claim 1, wherein sending the multimedia data to the control terminal comprises:
recording a screen of the display screen of the controlled terminal to obtain the multimedia data;
and sending the multimedia data to the control terminal.
4. The method according to claim 1, wherein the determining the display position of the trigger operation in the display screen of the controlled terminal according to the trigger position comprises:
and determining a display position corresponding to the trigger position according to a coordinate mapping relation, wherein the coordinate mapping relation is a mapping relation between a control terminal display screen coordinate system and a controlled terminal display screen coordinate system.
5. The method of any of claims 1 to 4, further comprising:
monitoring the playing progress of the multimedia data in the display screen of the controlled terminal;
and when the playing progress reaches a preset playing node, sending push information related to the multimedia data to the control terminal so that the control terminal outputs the push information to a display screen of the control terminal, and the push information is superposed on the multimedia data for displaying.
6. The method of any of claims 1 to 4, further comprising:
receiving a push request sent by the control terminal;
determining pushing information about the multimedia data according to the pushing request;
and sending the push information to the control terminal so that the control terminal outputs the push information to a display screen of the control terminal, and the push information is superposed on the multimedia data for display.
7. A terminal interaction method is applied to a control terminal and comprises the following steps:
receiving multimedia data sent by a controlled terminal;
acquiring a trigger operation aiming at the multimedia data;
generating an operation instruction according to the trigger operation, wherein the operation instruction comprises a trigger position and operation information of the trigger operation in a display screen of a control terminal;
and sending the operation instruction to the controlled terminal so that the controlled terminal determines the display position of the trigger operation in the display screen of the controlled terminal according to the trigger position and outputs the operation information at the display position.
8. The terminal interaction system is characterized by comprising a controlled terminal and a control terminal, wherein,
the control terminal is used for receiving the multimedia data sent by the controlled terminal, acquiring trigger operation aiming at the multimedia data, generating an operation instruction according to the trigger operation and sending the operation instruction to the controlled terminal;
the controlled terminal is used for sending the multimedia data to the control terminal, determining a trigger position and operation information of the trigger operation in a display screen of the control terminal according to the operation instruction, determining a display position of the trigger operation in the display screen of the controlled terminal according to the trigger position, and outputting the operation information at the display position.
9. An electronic device, comprising:
a memory for storing a computer program;
processor for implementing the steps of the terminal interaction method according to any of claims 1 to 6 or the steps of the terminal interaction method according to claim 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that a computer program is stored thereon, which, when being executed by a processor, carries out the steps of the terminal interaction method according to any one of claims 1 to 6 or the steps of the terminal interaction method according to claim 7.
CN202210706846.1A 2022-06-21 2022-06-21 Terminal interaction method, system, electronic equipment and computer readable storage medium Pending CN114979740A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210706846.1A CN114979740A (en) 2022-06-21 2022-06-21 Terminal interaction method, system, electronic equipment and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210706846.1A CN114979740A (en) 2022-06-21 2022-06-21 Terminal interaction method, system, electronic equipment and computer readable storage medium

Publications (1)

Publication Number Publication Date
CN114979740A true CN114979740A (en) 2022-08-30

Family

ID=82964769

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210706846.1A Pending CN114979740A (en) 2022-06-21 2022-06-21 Terminal interaction method, system, electronic equipment and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN114979740A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104038812A (en) * 2014-05-09 2014-09-10 海信集团有限公司 Information push method and device
CN106341735A (en) * 2015-07-07 2017-01-18 阿里巴巴集团控股有限公司 Information pushing method and device
CN111432284A (en) * 2020-04-30 2020-07-17 成都新潮传媒集团有限公司 Bullet screen interaction method of multimedia terminal and multimedia terminal
CN113485626A (en) * 2020-10-13 2021-10-08 青岛海信电子产业控股股份有限公司 Intelligent display device, mobile terminal and display control method
CN113810761A (en) * 2021-09-17 2021-12-17 上海哔哩哔哩科技有限公司 Multi-terminal interaction method, device and system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104038812A (en) * 2014-05-09 2014-09-10 海信集团有限公司 Information push method and device
CN106341735A (en) * 2015-07-07 2017-01-18 阿里巴巴集团控股有限公司 Information pushing method and device
CN111432284A (en) * 2020-04-30 2020-07-17 成都新潮传媒集团有限公司 Bullet screen interaction method of multimedia terminal and multimedia terminal
CN113485626A (en) * 2020-10-13 2021-10-08 青岛海信电子产业控股股份有限公司 Intelligent display device, mobile terminal and display control method
CN113810761A (en) * 2021-09-17 2021-12-17 上海哔哩哔哩科技有限公司 Multi-terminal interaction method, device and system

Similar Documents

Publication Publication Date Title
US20220312049A1 (en) Method for processing live-streaming interaction video and server
EP3242462B1 (en) Cooperative control method for user equipment, user equipment, and communication system
US9049341B2 (en) Method and apparatus for video resource management in video conference
KR102157634B1 (en) Image capturing method and local endpoint host device
CN111031368B (en) Multimedia playing method, device, equipment and storage medium
CN108769795B (en) Video signal realizing system and method
CN109803151B (en) Multimedia data stream switching method and device, storage medium and electronic device
CN112905289A (en) Application picture display method, device, terminal, screen projection system and medium
CN113329240A (en) Screen projection method and device
US9756096B1 (en) Methods for dynamically transmitting screen images to a remote device
CN111880865A (en) Multimedia data pushing method and device, electronic equipment and storage medium
CN114546308A (en) Application interface screen projection method, device, equipment and storage medium
CN114007125A (en) Volume control method, mobile terminal, target device and storage medium
CN112055252A (en) Multi-screen interaction method and device, computer readable medium and electronic equipment
CN113286190A (en) Cross-network and same-screen control method and device and cross-network and same-screen system
CN112153457A (en) Wireless screen projection connection method and device, computer storage medium and electronic equipment
CN107771387B (en) System and method for setting time and date in a device
CN111835988B (en) Subtitle generation method, server, terminal equipment and system
CN114979740A (en) Terminal interaction method, system, electronic equipment and computer readable storage medium
CN113099281B (en) Video interaction method and device, storage medium and terminal
CN115379279A (en) Multi-screen linkage interaction method, device and system, storage medium and electronic equipment
CN111356009B (en) Audio data processing method and device, storage medium and terminal
EP2463824A1 (en) Method and input-output device for rendering at least one of audio, video and computer graphics content and servicing device for delivering at least one of pre-rendered audio, pre-rendered video and pre-rendered computer graphics content
US10491681B2 (en) Method and a device for enriching a call
CN110794966B (en) AR display system and method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination