WO2021223453A1 - 多屏互动显示方法及装置 - Google Patents

多屏互动显示方法及装置 Download PDF

Info

Publication number
WO2021223453A1
WO2021223453A1 PCT/CN2020/141032 CN2020141032W WO2021223453A1 WO 2021223453 A1 WO2021223453 A1 WO 2021223453A1 CN 2020141032 W CN2020141032 W CN 2020141032W WO 2021223453 A1 WO2021223453 A1 WO 2021223453A1
Authority
WO
WIPO (PCT)
Prior art keywords
interactive
connection channel
audio
data
interactive data
Prior art date
Application number
PCT/CN2020/141032
Other languages
English (en)
French (fr)
Inventor
龙春宇
王昊
张瑶琦
杨智慧
王孟灰
黄磊
Original Assignee
上海幻电信息科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海幻电信息科技有限公司 filed Critical 上海幻电信息科技有限公司
Priority to US17/923,226 priority Critical patent/US11910072B2/en
Publication of WO2021223453A1 publication Critical patent/WO2021223453A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8545—Content authoring for generating interactive applications
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/488—Data services, e.g. news ticker
    • H04N21/4882—Data services, e.g. news ticker for displaying messages, e.g. warnings, reminders
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/488—Data services, e.g. news ticker
    • H04N21/4884—Data services, e.g. news ticker for displaying subtitles
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network

Definitions

  • This application relates to the field of Internet technology, and in particular to a multi-screen interactive display method, device, computer equipment, and computer-readable storage medium.
  • the electronic device can realize the switching and display of display content among multiple devices by means such as screen projection.
  • This application proposes a multi-screen interactive display method, device, computer equipment, and computer-readable storage medium, which can solve the problems of single projected screen content and poor interactivity in the prior art.
  • this application provides a multi-screen interactive display method, the method includes:
  • Receive a screen projection request from a screen projection terminal establish an audio and video data connection channel with the screen projection terminal; obtain audio and video data through the audio and video data connection channel and display it on the target display window; receive the screen projection terminal To obtain the interactive data type in the interactive request; establish an interactive data connection channel with the screen projection terminal according to the interactive data type; receive the interactive data of the screen projection terminal through the interactive data connection channel To display the interactive data on the target display window.
  • the acquiring audio and video data through the audio and video data connection channel includes: acquiring the resource address of the audio and video data in the screen projection request; and communicating with the audio and video data through the audio and video data connection channel.
  • the resource address connects and accesses the audio and video data.
  • the interactive data type includes at least one of text, picture, bullet screen, reward, and gift.
  • the establishing an interactive data connection channel with the screen projection terminal according to the interactive data type includes: querying the corresponding interactive data connection from a preset interactive data connection channel information table according to the interactive data type Channel type and establishment rule; the interactive data connection channel of the interactive data connection channel type is established according to the establishment rule.
  • the establishment rule includes establishing a data communication link through a preset protocol; the interactive data connection channel includes a communication link that supports the transmission of interactive data of at least one interactive data type.
  • the method further includes: sending the interactive data to an interactive server; receiving interactive feedback data from other terminals on the interactive data returned by the interactive server, and displaying the interactive feedback data to The target display window.
  • the method further includes: sending the interactive feedback data to the screen projection terminal through the interactive connection channel.
  • this application also provides a multi-screen interactive display device, which includes:
  • the first receiving module is used to receive a screen projection request from the screen projection terminal and establish an audio and video data connection channel with the screen projection terminal; the first display module is used to obtain audio and video data through the audio and video data connection channel The data is displayed on the target display window; the second receiving module is used to receive the interaction request of the screen projection terminal, and obtain the interactive data type in the interactive request; the establishment module is used to establish the interaction data type with the project The interactive data connection channel between screen terminals; the second display module is used to receive the interactive data of the screen projection terminal through the interactive data connection channel, and display the interactive data to the target display window.
  • this application also proposes a computer device, the computer device includes a memory and a processor, the memory stores computer-readable instructions that can run on the processor, and the computer-readable instructions are The following steps are implemented when the processor is executed:
  • Receive a screen projection request from a screen projection terminal establish an audio and video data connection channel with the screen projection terminal; obtain audio and video data through the audio and video data connection channel and display it on the target display window; receive the screen projection terminal To obtain the interactive data type in the interactive request; establish an interactive data connection channel with the screen projection terminal according to the interactive data type; receive the interactive data of the screen projection terminal through the interactive data connection channel To display the interactive data on the target display window.
  • the present application also provides a computer-readable storage medium, the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable The at least one processor performs the following steps:
  • Receive a screen projection request from a screen projection terminal establish an audio and video data connection channel with the screen projection terminal; obtain audio and video data through the audio and video data connection channel and display it on the target display window; receive the screen projection terminal To obtain the interactive data type in the interactive request; establish an interactive data connection channel with the screen projection terminal according to the interactive data type; receive the interactive data of the screen projection terminal through the interactive data connection channel To display the interactive data on the target display window.
  • the multi-screen interactive display method, device, computer equipment, and computer-readable storage medium proposed in this application can establish an audio and video data connection channel after receiving a screen projection request from a screen projection terminal, and obtain through the audio and video data connection channel The audio and video data is displayed on the target display window; then the interaction request of the projection terminal is received, the interactive data type obtained, and the interactive data connection channel with the projection terminal is established according to the interactive data type; and finally through The interactive data connection channel receives the interactive data of the screen projection terminal, and displays the interactive data on the target display window.
  • an interactive data connection channel independent of the audio and video data connection channel, it supports more types of screen projection content and realizes the interactive nature of the screen projection.
  • Fig. 1 is a schematic diagram of an application environment of an embodiment of the present application
  • FIG. 2 is a schematic flowchart of a specific embodiment of the multi-screen interactive display method of the present application
  • step S202 is a schematic flowchart of a specific embodiment of obtaining audio and video data through the audio and video data connection channel in step S202;
  • FIG. 4 is a schematic flowchart of a specific embodiment of establishing an interactive data connection channel with the projection terminal according to the interactive data type in step S206;
  • FIG. 5 is a schematic flowchart of a specific embodiment based on FIG. 2;
  • FIG. 6 is a flow effect diagram of a specific embodiment of the multi-screen interactive display method of the present application.
  • FIG. 7 is a schematic diagram of program modules of an embodiment of a multi-screen interactive display device according to the present application.
  • Fig. 8 is a schematic diagram of an optional hardware architecture of the computer device of the present application.
  • Fig. 1 is a schematic diagram of an application environment of an embodiment of the present application.
  • the computer device 1 is connected to the projection terminal 2 and the interactive server 3, and can receive the projection display request sent by the projection terminal 2, and establish audio and video data with the projection terminal 2.
  • the type establishes an interactive data connection channel with the screen projection terminal; finally, the interactive data of the screen projection terminal is received through the interactive data connection channel, and the interactive data is displayed on the target display window.
  • the screen projection terminal 2 may be a mobile phone, a tablet, a portable device or a PC, etc.
  • the computer device 1 may be a mobile phone, a tablet, a portable device, a PC, or an electronic device with a display function, etc.
  • FIG. 2 is a schematic flowchart of an embodiment of a multi-screen interactive display method according to the present application. It can be understood that the flowchart in this method embodiment is not used to limit the order of execution of the steps. The following is an exemplary description with the computer device 1 as the execution subject.
  • the multi-screen interactive display method may include steps S200 to S208, wherein:
  • Step S200 Receive a screen projection request from a screen projection terminal, and establish an audio and video data connection channel with the screen projection terminal.
  • Step S202 Acquire audio and video data through the audio and video data connection channel and display it on a target display window.
  • the projection terminal when the user wants to screen the audio and video data on the projection terminal onto the computer device 1, the projection terminal first needs to connect to the computer device 1, and then the user sends the video data to the computer device 1 through the projection terminal.
  • the computer device 1 sends a screen projection request.
  • the computer device 1 receives the screen projection request, and then establishes an audio and video data connection channel according to the screen projection request.
  • obtaining audio and video data through the audio and video data connection channel in step S202 includes steps S300 to S302.
  • Step S300 Obtain the resource address of the audio and video data in the screen projection request.
  • Step S302 Connect to the resource address through the audio and video data connection channel, and access the audio and video data.
  • the projection terminal needs to project the video data on demand, live broadcast, or real-time playback to the computer device 1. Therefore, the projection terminal needs to project the format of the audio and video data to be projected and
  • the resource path or address is encapsulated into a screen projection request and sent to the computer device 1. Since the audio and video data connection channels generally support most of the audio and video formats, the computer device 1 can directly establish the audio and video data connection channel after receiving the screen projection request for transmitting the content of the screen projection.
  • the audio and video data is then connected to the resource address or path through the audio and video data connection channel, and the audio and video data is accessed, and the audio and video data can be called and displayed on the target display window.
  • a screen projection terminal orders video a from resource address A.
  • the screen projection terminal wants to project the on-demand video a to the computer device 1
  • the screen projection terminal sends the screen projection to the computer device 1.
  • Request the computer device 1 establishes an audio and video data connection channel according to the screen projection request, and then obtains the resource address A from the screen projection request, and finally connects to the resource address A on the screen projection terminal through the audio and video data connection channel , And then access and play video a.
  • the computer device 1 can directly connect to the shooting unit of the projection terminal through the audio and video data connection channel, and obtain the projection in real time. The audio and video data shot by the shooting unit of the terminal for playback.
  • Step S204 Receive an interaction request from the screen projection terminal, and obtain the interaction data type in the interaction request.
  • Step S206 Establish an interactive data connection channel with the projection terminal according to the interactive data type.
  • the user can also trigger an interactive message through the screen projection terminal.
  • the screen projection terminal sends an interaction request to the computer device 1.
  • the computer device 1 acquires the interaction data type in the interaction request.
  • the interactive data type includes at least one of text, picture, bullet screen, reward, and gift.
  • the computer device 1 further establishes an interactive data connection channel with the projection terminal according to the interactive data type, and the interactive data connection channel is used to transmit the above interactive data. That is, the interactive data connection channel is a second connection channel different from the audio and video data connection channel.
  • establishing an interactive data connection channel with the screen projection terminal according to the interactive data type in step S206 includes steps S400 to S402.
  • Step S400 Query the corresponding interactive data connection channel type and establishment rule from the preset interactive data connection channel information table according to the interactive data type.
  • Step S402 Establish an interactive data connection channel of the interactive data connection channel type according to the establishment rule.
  • the computer device 1 and the projection terminal are pre-configured with an interactive data connection channel information table including rules for establishing interactive data connection channels for different interactive data. Therefore, after the computer device 1 obtains the interactive data type, it will query the corresponding interactive data connection channel type and establishment rules from the preset interactive data connection channel information table according to the interactive data type, and then according to The establishment rule establishes an interactive data connection channel of the interactive data connection channel type.
  • the establishment rule includes establishing a data communication link through a preset protocol; the interactive data connection channel includes a communication link that supports the transmission of interactive data of at least one interactive data type.
  • the general audio and video data connection data channel is based on the UDP protocol (User Datagram Protocol, user datagram protocol), and only supports the data link for audio and video data transmission. Therefore, the computer device 1 can establish a TCP protocol (Transmission Control Protocol, Transmission Control Protocol) supporting barrage, or a data link of interactive data types such as rewards, that is, an interactive data connection channel.
  • TCP protocol Transmission Control Protocol, Transmission Control Protocol
  • the data format that supports transmission included in the creation rule of the interactive data connection channel corresponds to the format of the interactive data type.
  • Step S208 Receive the interactive data of the projection terminal through the interactive data connection channel, and display the interactive data on the target display window.
  • the computer device 1 can receive the interactive data sent by the projection terminal through the interactive data connection channel, and then display the interactive data on the video playback screen of the target display window.
  • the multi-screen interactive display method may include steps S500-S502 in addition to steps S200-S208.
  • Step S500 Send the interactive data to the interactive server.
  • Step S502 Receive interactive feedback data from other terminals on the interactive data returned by the interactive server, and display the interactive feedback data on the target display window.
  • the computer device 1 since the interactive messages of multiple terminal devices for the same playing video need to be transferred to each other, after the computer device 1 obtains the interactive data sent by the screen projection terminal, it will forward it to the interactive server, where the interactive The server stores all interactive data of the same video, and the interactive server will push the interactive data to other video playback terminals after receiving the interactive data. Then, the computer device 1 can also receive interactive feedback data from other terminals on the interactive data returned on the interactive server, and display the interactive feedback data on the target display window.
  • the computer device 1 displays the interactive feedback data on the target display window, it will also send the interactive feedback data to the screen projection terminal through the interactive connection channel.
  • FIG. 6 it is a flow chart of a specific embodiment. Among them: First, the user triggers the interaction on the small-screen mobile phone, and then the small-screen mobile phone encapsulates the interactive data according to the interaction triggered by the user; the small-screen mobile phone sends the encapsulated interactive data to the large-screen device (ie, computer device 1) through the interactive data connection channel . Then, the large-screen device receives and parses the interactive data, and displays the interactive data to the video playback window. At the same time, the large-screen device sends the interactive data to the back-end interactive system (ie, the interactive server), and the back-end interactive system receives, analyzes, reviews, and processes the interactive data.
  • the back-end interactive system ie, the interactive server
  • the background interactive system is to check the interactive data, modify or delete the unqualified interactive data appropriately, and store the qualified interactive data. Then the large-screen device will also determine whether the background interactive system returns feedback data. When the background interactive system returns feedback data, the large-screen device will display and process the feedback data; at the same time, the feedback data will be sent to the small-screen mobile phone through the interactive data connection channel. Finally, the small screen mobile phone receives the feedback data, and then waits for the user to initiate a new interaction.
  • the multi-screen interactive display method proposed in this embodiment can establish an audio and video data connection channel after receiving a screen projection request from a screen projection terminal, and obtain audio and video data through the audio and video data connection channel and display it to The target display window; and then receive the interaction request of the screen projection terminal, the acquired interactive data type, and establish an interactive data connection channel with the screen projection terminal according to the interactive data type; finally through the interactive data connection channel Receiving the interactive data of the projection terminal, and displaying the interactive data on the target display window.
  • This application establishes an interactive data connection channel that is independent of the audio and video data connection channel. Compared with the current small-screen device casting to a large-screen device, it is generally mirrored or the small-screen device directly transmits the audio and video. The way the data is sent to the large-screen device for playback can support more types of screencast content and realize the interactive nature of the screencast.
  • FIG. 7 schematically shows a block diagram of a multi-screen interactive display device according to the second embodiment of the present application.
  • the multi-screen interactive display device can be divided into one or more program modules, and the one or more program modules are stored in a storage medium , And executed by one or more processors to complete the embodiments of the present application.
  • the program module referred to in the embodiment of the present application refers to a series of computer-readable instruction segments that can complete specific functions. The following description will specifically introduce the function of each program module in this embodiment.
  • the multi-screen interactive display device 400 may include a first receiving module 410, a first display module 420, a second receiving module 430, a establishing module 440, and a second display module 450, wherein:
  • the first receiving module 410 is configured to receive a screen projection request from a screen projection terminal, and establish an audio and video data connection channel with the screen projection terminal.
  • the first display module 420 is configured to obtain audio and video data through the audio and video data connection channel and display it on the target display window.
  • the second receiving module 430 is configured to receive the interaction request of the projection terminal, and obtain the interaction data type in the interaction request.
  • the establishment module 440 is configured to establish an interactive data connection channel with the projection terminal according to the interactive data type.
  • the second display module 450 is configured to receive the interactive data of the projection terminal through the interactive data connection channel, and display the interactive data on the target display window.
  • the first display module 420 is further configured to: obtain the resource address of the audio and video data in the screen projection request; communicate with the resource address through the audio and video data connection channel Connect and access the audio and video data.
  • the establishment module 430 is further configured to: query the corresponding interactive data connection channel type and establishment rule from the preset interactive data connection channel information table according to the interactive data type; An interactive data connection channel of the interactive data connection channel type is established.
  • the interactive data type includes at least one of text, picture, bullet screen, reward, and gift;
  • the establishment rule includes establishing a data communication link through a preset protocol;
  • the interactive data connection channel includes supporting at least one A communication link for the transmission of interactive data of a type of interactive data.
  • the second display module 450 is further configured to: send the interactive data to an interactive server; receive interactive feedback data from other terminals on the interactive data returned by the interactive server, and Displaying the interactive feedback data on the target display window; and sending the interactive feedback data to the screen projection terminal through the interactive connection channel.
  • FIG. 8 schematically shows a schematic diagram of the hardware architecture of a computer device 1 suitable for implementing the method for multi-screen interactive display according to the third embodiment of the present application.
  • the computer device 1 is a device that can automatically perform numerical calculation and/or information processing in accordance with pre-set or stored instructions.
  • it may be a rack server, a blade server, a tower server or a cabinet server (including an independent server or a server cluster composed of multiple servers) with a gateway function.
  • the computer device 1 at least includes but is not limited to: a memory 510, a processor 520, and a network interface 530 that can communicate with each other through a system bus. in:
  • the memory 510 includes at least one type of computer-readable storage medium.
  • the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), and static random access memory.
  • SRAM read-only memory
  • ROM read-only memory
  • EEPROM electrically erasable programmable read-only memory
  • PROM programmable read-only memory
  • magnetic memory magnetic disk, optical disk, etc.
  • the memory 510 may be an internal storage module of the computer device 1, such as a hard disk or memory of the computer device 1.
  • the memory 510 may also be an external storage device of the computer device 1, such as a plug-in hard disk equipped on the computer device 1, a smart memory card (Smart Media Card, referred to as SMC), and a secure digital (Secure Digital). Digital, abbreviated as SD) card, flash card (Flash Card), etc.
  • the memory 510 may also include both an internal storage module of the computer device 1 and an external storage device thereof.
  • the memory 510 is generally used to store an operating system and various application software installed in the computer device 1, such as program codes of a multi-screen interactive display method.
  • the memory 510 may also be used to temporarily store various types of data that have been output or will be output.
  • the processor 520 may be a central processing unit (Central Processing Unit, CPU for short), a controller, a microcontroller, a microprocessor, or other data processing chips.
  • the processor 520 is generally used to control the overall operation of the computer device 1, such as performing data interaction or communication-related control and processing with the computer device 1.
  • the processor 520 is configured to run program codes stored in the memory 510 or process data.
  • the network interface 530 may include a wireless network interface or a wired network interface, and the network interface 530 is generally used to establish a communication link between the computer device 1 and other computer devices.
  • the network interface 530 is used to connect the computer device 1 with an external terminal through a network, and establish a data transmission channel and a communication link between the computer device 1 and the external terminal.
  • the network can be Intranet, Internet, Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network , 5G network, Bluetooth (Bluetooth), Wi-Fi and other wireless or wired networks.
  • FIG. 8 only shows a computer device with components 510-530, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.
  • the program code of the multi-screen interactive display method stored in the memory 510 can also be divided into one or more program modules, which are executed by one or more processors (in this embodiment, the processor 520). Execute to complete the embodiments of this application.
  • This embodiment also provides a computer-readable storage medium on which computer-readable instructions are stored, and when the computer-readable instructions are executed by a processor, the following steps are implemented:
  • Receive a screen projection request from a screen projection terminal establish an audio and video data connection channel with the screen projection terminal; obtain audio and video data through the audio and video data connection channel and display it on the target display window; receive the screen projection terminal To obtain the interactive data type in the interactive request; establish an interactive data connection channel with the screen projection terminal according to the interactive data type; receive the interactive data of the screen projection terminal through the interactive data connection channel To display the interactive data on the target display window.
  • the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disks, optical disks, etc.
  • the computer-readable storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device.
  • the computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, referred to as SMC), and a secure digital ( Secure Digital, referred to as SD card, Flash Card, etc.
  • the computer-readable storage medium may also include both the internal storage unit and the external storage device of the computer device.
  • the computer-readable storage medium is generally used to store the operating system and various application software installed in the computer device, such as the program code of the component management method of the service platform in the embodiment.
  • the computer-readable storage medium can also be used to temporarily store various types of data that have been output or will be output.
  • modules or steps of the embodiments of the present application described above can be implemented by a general computing device, and they can be concentrated on a single computing device or distributed among multiple computing devices.
  • they can be implemented by the program code executable by the computing device, so that they can be stored in the storage device for execution by the computing device, and in some cases, they can be different from here
  • the steps shown or described are executed in the order of, or they are respectively fabricated into individual integrated circuit modules, or multiple modules or steps of them are fabricated into a single integrated circuit module to achieve. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Computer Security & Cryptography (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Between Computers (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请公开了一种多屏互动显示方法，该方法包括：接收投屏终端的投屏请求，建立与所述投屏终端之间的音视频数据连接通道；通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口；接收所述投屏终端的互动请求，获取所述互动请求中的互动数据类型；根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。本申请还提供一种多屏互动显示装置、计算机设备以及计算机可读存储介质。本申请通过建立与音视频数据连接通道相互独立的互动数据连接通道，从而支持更多类型的投屏内容，实现了投屏的互动性。

Description

多屏互动显示方法及装置
本申请要求于2020年05月06日提交中国专利局、申请号为202010374621.1、发明名称为“多屏互动显示方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及互联网技术领域，尤其涉及一种多屏互动显示方法、装置、计算机设备及计算机可读存储介质。
背景技术
随着计算机技术的发展，越来越多种类的电子设备被开发并被广泛使用。各类电子设备一般具有各自的设备特点，例如，手机的便携性更好，电视屏幕的显示效果更好，而音箱的音质效果更好。为了充分发挥不同电子设备的设备特点，电子设备可以通过投屏等方式实现显示内容在多个设备之间的切换和显示。
然而，发明人意识到，现有的投屏技术仅支持音视频的投屏，不能展现其他互动数据，因此，投屏的内容比较单一，互动性差。
发明内容
本申请提出一种多屏互动显示方法、装置、计算机设备及计算机可读存储介质，能够解决现有技术中投屏内容单一，互动性差的问题。
首先，为实现上述目的，本申请提供一种多屏互动显示方法，所述方法包括：
接收投屏终端的投屏请求，建立与所述投屏终端之间的音视频数据连接通道；通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口；接收所述投屏终端的互动请求，获取所述互动请求中的互动数据类型；根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。
在一个例子中，所述通过所述音视频数据连接通道获取音视频数据包括：获取所述投屏请求中的所述音视频数据的资源地址；透过所述音视频数据连接通道与所述资源地址进行连接，并访问所述音视频数据。
在一个例子中，所述互动数据类型包括文字、图片、弹幕、打赏、礼物中的至少一种。
在一个例子中，所述根据互动数据类型建立与所述投屏终端之间的互动数据连接通道包括：根据所述互动数据类型从预设的互动数据连接通道信息表中查询对应的互动数据连接通道类型及建立规则；根据所述建立规则建立所述互动数据连接通道类型的互动数据连接通道。
在一个例子中，所述建立规则包括通过预设协议建立数据通信链路；所述互动数据连接通道包括支持至少一种互动数据类型的互动数据进行传输的通信链路。
在一个例子中，所述方法还包括：将所述互动数据发送至互动服务器；接收由所述互动服务器返回的其他终端对于所述互动数据的互动反馈数据，并将所述互动反馈数据显示到所述目标显示窗口。
在一个例子中，所述方法还包括：将所述互动反馈数据通过所述互动连接通道发送至所述投屏终端。
此外，为实现上述目的，本申请还提供一种多屏互动显示装置，所述装置包括：
第一接收模块，用于接收投屏终端的投屏请求，建立与所述投屏终端之间的音视频数据连接通道；第一显示模块，用于通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口；第二接收模块，用于接收所述投屏终端的互动请求，获取所述互动请求中的互动数据类型；建立模块，用于根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；第二显示模块，用于通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。
进一步地，本申请还提出一种计算机设备，所述计算机设备包括存储器、处理器，所述存储器上存储有可在所述处理器上运行的计算机可读指令，所述计算机可读指令被所述处理器执行时实现以下步骤：
接收投屏终端的投屏请求，建立与所述投屏终端之间的音视频数据连接通道；通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口；接收所述投屏终端的互动请求，获取所述互动请求中的互动数据类型；根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。
进一步地，为实现上述目的，本申请还提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机可读指令，所述计算机可读指令可被至少一个处理器执行，以使所述至少一个处理器执行以下步骤：
接收投屏终端的投屏请求，建立与所述投屏终端之间的音视频数据连接通道；通过所 述音视频数据连接通道获取音视频数据并显示到目标显示窗口；接收所述投屏终端的互动请求，获取所述互动请求中的互动数据类型；根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。
本申请所提出的多屏互动显示方法、装置、计算机设备及计算机可读存储介质，能够在接收投屏终端的投屏请求之后，建立音视频数据连接通道，通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口；然后再接收所述投屏终端的互动请求，获取的互动数据类型，并根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；最后通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。通过建立与音视频数据连接通道相互独立的互动数据连接通道，从而支持更多类型的投屏内容，实现了投屏的互动性。
附图说明
图1是本申请一实施例的应用环境示意图；
图2是本申请多屏互动显示方法一具体实施例的流程示意图；
图3是步骤S202中的通过所述音视频数据连接通道获取音视频数据的一具体实施例的流程示意图；
图4是步骤S206中的根据互动数据类型建立与所述投屏终端之间的互动数据连接通道的一具体实施例的流程示意图；
图5是基于图2的一具体实施例的流程示意图；
图6是本申请多屏互动显示方法一具体实施例的流程效果图；
图7是本申请多屏互动显示装置一实施例的程序模块示意图；
图8是本申请计算机设备一可选的硬件架构的示意图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
需要说明的是，在本申请中涉及“第一”、“第二”等的描述仅用于描述目的，而不能理解为指示或暗示其相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第 一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。另外，各个实施例之间的技术方案可以相互结合，但是必须是以本领域普通技术人员能够实现为基础，当技术方案的结合出现相互矛盾或无法实现时应当认为这种技术方案的结合不存在，也不在本申请要求的保护范围之内。
图1是本申请一实施例的应用环境示意图。参阅图1所示，所述计算机设备1与投屏终端2以及互动服务器3连接，可以接收由投屏终端2发送的投屏显示请求，建立与所述投屏终端2之间的音视频数据连接通道，并通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口；然后再接收所述投屏终端2的互动请求，获取所述互动请求中的互动数据类型，根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；最后通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。在本实施例中，所述投屏终端2可以是手机、平板、便携设备或者PC机等；所述计算机设备1可作为手机、平板、便携设备、PC机或者具有显示功能的电子设备等。
实施例一
图2是本申请多屏互动显示方法一实施例的流程示意图。可以理解，本方法实施例中的流程图不用于对执行步骤的顺序进行限定。下面以计算机设备1为执行主体进行示例性描述。
如图2所示，所述多屏互动显示方法可以包括步骤S200～S208其中：
步骤S200，接收投屏终端的投屏请求，建立与所述投屏终端之间的音视频数据连接通道。
步骤S202，通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口。
具体地，当用户想要将投屏终端上的音视频数据投屏到所述计算机设备1上时，那么投屏终端首先需要与所述计算机设备1进行网络连接，然后用户通过投屏终端向所述计算机设备1发送投屏请求。所述计算机设备1则接收所述投屏请求，然后根据所述投屏请求建立音视频数据连接通道。
如图3所示，在一示例性的实施例中，步骤S202中的通过所述音视频数据连接通道获取音视频数据，包括步骤S300～S302。
步骤S300，获取所述投屏请求中的所述音视频数据的资源地址。
步骤S302，透过所述音视频数据连接通道与所述资源地址进行连接，并访问所述音视频数据。
在本实施例中，所述投屏终端需要将点播、直播或者实时播放的视频数据投屏到所述 计算机设备1，因此，所述投屏终端需要将待投屏的音视频数据的格式以及资源路径或地址封装成投屏请求发送给所述计算机设备1。由于音视频数据连接通道一般都支持绝大多数的音频格式和视频格式，因此所述计算机设备1接收到投屏请求后，则可以直接建立音视频数据连接通道，用于传输投屏内容中的音视频数据，然后透过所述音视频数据连接通道与所述资源地址或路径进行连接，并访问所述音视频数据，也就可以调用所述音视频数据，显示到目标显示窗口。
例如，投屏终端从资源地址A点播了视频a，当投屏终端想要将点播的视频a投屏到所述计算机设备1，那么，所述投屏终端向所述计算机设备1发送投屏请求，所述计算机设备1根据投屏请求建立音视频数据连接通道，然后再从投屏请求中获取到资源地址A，最后透过音视频数据连接通道与投屏终端连接上的资源地址A连接，然后访问并播放视频a。再例如，投屏终端直接将拍摄的音视频数据进行投屏时，那么所述计算机设备1则可以透过所述音视频数据连接通道直接与投屏终端的拍摄单元连接，并实时获取投屏终端的拍摄单元所拍摄的音视频数据，以进行播放。
步骤S204，接收所述投屏终端的互动请求，获取所述互动请求中的互动数据类型。
步骤S206，根据互动数据类型建立与所述投屏终端之间的互动数据连接通道。
具体地，在投屏的过程中，用户还可以通过投屏终端触发互动消息。当用户通过投屏终端触发互动消息时，例如，用户通过投屏终端进行弹幕或者打赏，那么，投屏终端则会向所述计算机设备1发送互动请求。所述计算机设备1接收到所述互动请求后，则获取所述互动请求中的互动数据类型。在本实施例中，所述互动数据类型包括文字、图片、弹幕、打赏、礼物中的至少一种。接着，所述计算机设备1进一步根据互动数据类型建立与所述投屏终端之间的互动数据连接通道，所述互动数据连接通道用于传输以上所述互动数据。也就是说，所述互动数据连接通道是不同于所述音视频数据连接通道的第二连接通道。
如图4所示，在一示例性的实施例中，步骤S206中的根据互动数据类型建立与所述投屏终端之间的互动数据连接通道，包括步骤S400～S402。
步骤S400，根据所述互动数据类型从预设的互动数据连接通道信息表中查询对应的互动数据连接通道类型及建立规则。
步骤S402，根据所述建立规则建立所述互动数据连接通道类型的互动数据连接通道。
具体地，所述计算机设备1与投屏终端预先配置好了包括对于不同互动数据的互动数据连接通道建立规则的互动数据连接通道信息表。因此，当所述计算机设备1获取到所述互动数据类型之后，则会根据所述互动数据类型从预设的互动数据连接通道信息表中查询对应的互动数据连接通道类型及建立规则，然后根据所述建立规则建立所述互动数据连接 通道类型的互动数据连接通道。在本实施例中，所述建立规则包括通过预设协议建立数据通信链路；所述互动数据连接通道包括支持至少一种互动数据类型的互动数据进行传输的通信链路。例如，一般的音视频数连接据通道为基于UDP协议(User Datagram Protocol，用户数据报协议)，仅支持音视频数据传输的数据链路。因此，所述计算机设备1可以建立TCP协议(Transmission Control Protocol，传输控制协议)的支持弹幕，或者打赏等互动数据类型的数据链路，也就是互动数据连接通道。当然，互动数据连接通道的创建规则中包括的支持传输的数据格式则对应于互动数据类型的格式。
步骤S208，通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。
当建立好互动数据连接通道之后，所述计算机设备1则可以通过所述互动数据连接通道接收投屏终端发送的互动数据，然后将互动数据显示到所述目标显示窗口的视频播放画面。
如图5所示，在一示例性的实施例中，所述多屏互动显示方法除了包括步骤S200～S208，还可以包括步骤S500～S502。
步骤S500，将所述互动数据发送至互动服务器。
步骤S502，接收由所述互动服务器返回的其他终端对于所述互动数据的互动反馈数据，并将所述互动反馈数据显示到所述目标显示窗口。
具体地，由于多个终端设备对于同一播放视频进行的互动消息需要相互传递，因此，所述计算机设备1获取到投屏终端发送的互动数据之后，还会转发至互动服务器，其中，所述互动服务器存储同一视频的所有互动数据，所述互动服务器接收到所述互动数据后还会推送到其他视频播放终端。然后，所述计算机设备1还能够接收所述互动服务器上返回的其他终端对于所述互动数据的互动反馈数据，并将所述互动反馈数据显示到所述目标显示窗口。
当然，所述计算机设备1将所述互动反馈数据显示到所述目标显示窗口之后，还会将所述互动反馈数据通过所述互动连接通道发送至所述投屏终端。
如图6所示，是一具体实施例的流程效果图。其中：首先，用户在小屏手机上触发互动，然后小屏手机根据用户触发的互动封装互动数据；小屏手机将封装的互动数据通过互动数据连接通道发送给大屏设备(即计算机设备1)。接着，大屏设备接收并解析互动数据，并将互动数据展示到视频播放窗口。同时，大屏设备将互动数据发送至后台互动系统(即互动服务器)，后台互动系统接收、解析、审查及处理互动数据。后台互动系统也就是对互动数据进行检查，对于不合格的互动数据进行适当修改或删除，对于合格的互动数据则进行 存储。然后大屏设备还会判断所述后台互动系统是否返回反馈数据。当后台互动系统返回反馈数据时，大屏设备则会对反馈数据进行显示处理；同时还会将反馈数据通过互动数据连接通道发送给小屏手机。最后，小屏手机接收反馈数据，然后等待用户发起新的互动。
综上所述，本实施例所提出的多屏互动显示方法能够在接收投屏终端的投屏请求之后，建立音视频数据连接通道，通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口；然后再接收所述投屏终端的互动请求，获取的互动数据类型，并根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；最后通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。本申请通过建立与音视频数据连接通道相互独立的互动数据连接通道，对比于目前小屏设备向大屏设备进行投屏时，一般都是以镜像的方式投屏或者小屏设备直接将音视频数据发送至大屏设备进行播放的方式，能够支持更多类型的投屏内容，实现了投屏的互动性。
实施例二
图7示意性示出了根据本申请实施例二的多屏互动显示装置的框图，该多屏互动显示装置可以被分割成一个或多个程序模块，一个或者多个程序模块被存储于存储介质中，并由一个或多个处理器所执行，以完成本申请实施例。本申请实施例所称的程序模块是指能够完成特定功能的一系列计算机可读指令段，以下描述将具体介绍本实施例中各程序模块的功能。
如图7所示，该多屏互动显示装置400可以包括第一接收模块410、第一显示模块420、第二接收模块430、建立模块440和第二显示模块450，其中：
第一接收模块410，用于接收投屏终端的投屏请求，建立与所述投屏终端之间的音视频数据连接通道。
第一显示模块420，用于通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口。
第二接收模块430，用于接收所述投屏终端的互动请求，获取所述互动请求中的互动数据类型。
建立模块440，用于根据互动数据类型建立与所述投屏终端之间的互动数据连接通道。
第二显示模块450，用于通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。
在示例性的实施例中，第一显示模块420，还用于：获取所述投屏请求中的所述音视频数据的资源地址；透过所述音视频数据连接通道与所述资源地址进行连接，并访问所述 音视频数据。
在示例性的实施例中，建立模块430，还用于：根据所述互动数据类型从预设的互动数据连接通道信息表中查询对应的互动数据连接通道类型及建立规则；根据所述建立规则建立所述互动数据连接通道类型的互动数据连接通道。其中，所述互动数据类型包括文字、图片、弹幕、打赏、礼物中的至少一种；所述建立规则包括通过预设协议建立数据通信链路；所述互动数据连接通道包括支持至少一种互动数据类型的互动数据进行传输的通信链路。
在示例性的实施例中，第二显示模块450，还用于：将所述互动数据发送至互动服务器；接收由所述互动服务器返回的其他终端对于所述互动数据的互动反馈数据，并将所述互动反馈数据显示到所述目标显示窗口；以及将所述互动反馈数据通过所述互动连接通道发送至所述投屏终端。
实施例三
图8示意性示出了根据本申请实施例三的适于实现多屏互动显示方法的计算机设备1的硬件架构示意图。本实施例中，计算机设备1是一种能够按照事先设定或者存储的指令，自动进行数值计算和/或信息处理的设备。例如，可以是具有网关功能的机架式服务器、刀片式服务器、塔式服务器或机柜式服务器(包括独立的服务器，或者多个服务器所组成的服务器集群)等。如图8所示，计算机设备1至少包括但不限于：可通过系统总线相互通信链接存储器510、处理器520、网络接口530。其中：
存储器510至少包括一种类型的计算机可读存储介质，可读存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如，SD或DX存储器等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘等。在一些实施例中，存储器510可以是计算机设备1的内部存储模块，例如该计算机设备1的硬盘或内存。在另一些实施例中，存储器510也可以是计算机设备1的外部存储设备，例如该计算机设备1上配备的插接式硬盘，智能存储卡(Smart Media Card，简称为SMC)，安全数字(Secure Digital，简称为SD)卡，闪存卡(Flash Card)等。当然，存储器510还可以既包括计算机设备1的内部存储模块也包括其外部存储设备。本实施例中，存储器510通常用于存储安装于计算机设备1的操作系统和各类应用软件，例如多屏互动显示方法的程序代码等。此外，存储器510还可以用于暂时地存储已经输出或者将要输出的各类数据。
处理器520在一些实施例中可以是中央处理器(Central Processing Unit，简称为CPU)、 控制器、微控制器、微处理器、或其他数据处理芯片。该处理器520通常用于控制计算机设备1的总体操作，例如执行与计算机设备1进行数据交互或者通信相关的控制和处理等。本实施例中，处理器520用于运行存储器510中存储的程序代码或者处理数据。
网络接口530可包括无线网络接口或有线网络接口，该网络接口530通常用于在计算机设备1与其他计算机设备之间建立通信链接。例如，网络接口530用于通过网络将计算机设备1与外部终端相连，在计算机设备1与外部终端之间的建立数据传输通道和通信链接等。网络可以是企业内部网(Intranet)、互联网(Internet)、全球移动通讯系统(Global System of Mobile communication，简称为GSM)、宽带码分多址(Wideband Code Division Multiple Access，简称为WCDMA)、4G网络、5G网络、蓝牙(Bluetooth)、Wi-Fi等无线或有线网络。
需要指出的是，图8仅示出了具有部件510-530的计算机设备，但是应理解的是，并不要求实施所有示出的部件，可以替代的实施更多或者更少的部件。
在本实施例中，存储于存储器510中的多屏互动显示方法的程序代码还可以被分割为一个或者多个程序模块，并由一个或多个处理器(本实施例为处理器520)所执行，以完成本申请实施例。
实施例四
本实施例还提供一种计算机可读存储介质，计算机可读存储介质其上存储有计算机可读指令，计算机可读指令被处理器执行时实现以下步骤：
接收投屏终端的投屏请求，建立与所述投屏终端之间的音视频数据连接通道；通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口；接收所述投屏终端的互动请求，获取所述互动请求中的互动数据类型；根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。
本实施例中，计算机可读存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如，SD或DX存储器等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘等。在一些实施例中，计算机可读存储介质可以是计算机设备的内部存储单元，例如该计算机设备的硬盘或内存。在另一些实施例中，计算机可读存储介质也可以是计算机设备的外部存储设备，例如该计算机设备上配备的插接式硬盘，智能存储卡(Smart Media Card，简称为SMC)，安全数字(Secure Digital，简称为SD)卡，闪存卡 (Flash Card)等。当然，计算机可读存储介质还可以既包括计算机设备的内部存储单元也包括其外部存储设备。本实施例中，计算机可读存储介质通常用于存储安装于计算机设备的操作系统和各类应用软件，例如实施例中服务平台的组件管理方法的程序代码等。此外，计算机可读存储介质还可以用于暂时地存储已经输出或者将要输出的各类数据。
显然，本领域的技术人员应该明白，上述的本申请实施例的各模块或各步骤可以用通用的计算装置来实现，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上，可选地，它们可以用计算装置可执行的程序代码来实现，从而，可以将它们存储在存储装置中由计算装置来执行，并且在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本申请实施例不限制于任何特定的硬件和软件结合。
以上仅为本申请实施例的优选实施例，并非因此限制本申请实施例的专利范围，凡是利用本申请实施例说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请实施例的专利保护范围内。

Claims (20)

  1. 一种多屏互动显示方法，所述方法包括：
    接收投屏终端的投屏请求，建立与所述投屏终端之间的音视频数据连接通道；
    通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口；
    接收所述投屏终端的互动请求，获取所述互动请求中的互动数据类型；
    根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；
    通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。
  2. 如权利要求1所述的多屏互动显示方法，所述通过所述音视频数据连接通道获取音视频数据包括：
    获取所述投屏请求中的所述音视频数据的资源地址；
    透过所述音视频数据连接通道与所述资源地址进行连接，并访问所述音视频数据。
  3. 如权利要求1所述的多屏互动显示方法，所述互动数据类型包括文字、图片、弹幕、打赏、礼物中的至少一种。
  4. 如权利要求1所述的多屏互动显示方法，所述根据互动数据类型建立与所述投屏终端之间的互动数据连接通道包括：
    根据所述互动数据类型从预设的互动数据连接通道信息表中查询对应的互动数据连接通道类型及建立规则；
    根据所述建立规则建立所述互动数据连接通道类型的互动数据连接通道。
  5. 如权利要求4所述的多屏互动显示方法，所述建立规则包括通过预设协议建立数据通信链路；所述互动数据连接通道包括支持至少一种互动数据类型的互动数据进行传输的通信链路。
  6. 如权利要求1所述的多屏互动显示方法，所述方法还包括：
    将所述互动数据发送至互动服务器；
    接收由所述互动服务器返回的其他终端对于所述互动数据的互动反馈数据，并将所述互动反馈数据显示到所述目标显示窗口。
  7. 如权利要求6所述的多屏互动显示方法，所述方法还包括：将所述互动反馈数据通过所述互动连接通道发送至所述投屏终端。
  8. 一种多屏互动显示装置，所述装置包括：
    第一接收模块，用于接收投屏终端的投屏请求，建立与所述投屏终端之间的音视频数据连接通道；
    第一显示模块，用于通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口；
    第二接收模块，用于接收所述投屏终端的互动请求，获取所述互动请求中的互动数据类型；
    建立模块，用于根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；
    第二显示模块，用于通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。
  9. 一种计算机设备，所述计算机设备包括存储器、处理器，所述存储器上存储有可在所述处理器上运行的计算机可读指令，所述计算机可读指令被所述处理器执行时实现以下步骤：
    接收投屏终端的投屏请求，建立与所述投屏终端之间的音视频数据连接通道；
    通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口；
    接收所述投屏终端的互动请求，获取所述互动请求中的互动数据类型；
    根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；
    通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。
  10. 如权利要求9所述的计算机设备，所述通过所述音视频数据连接通道获取音视频数据包括：
    获取所述投屏请求中的所述音视频数据的资源地址；
    透过所述音视频数据连接通道与所述资源地址进行连接，并访问所述音视频数据。
  11. 如权利要求9所述的计算机设备，所述根据互动数据类型建立与所述投屏终端之间的互动数据连接通道包括：
    根据所述互动数据类型从预设的互动数据连接通道信息表中查询对应的互动数据连接通道类型及建立规则；
    根据所述建立规则建立所述互动数据连接通道类型的互动数据连接通道。
  12. 如权利要求11所述的计算机设备，所述建立规则包括通过预设协议建立数据通信链路；所述互动数据连接通道包括支持至少一种互动数据类型的互动数据进行传输的通信链路。
  13. 如权利要求9所述的计算机设备，所述计算机可读指令被所述处理器执行时还实现以下步骤：
    将所述互动数据发送至互动服务器；
    接收由所述互动服务器返回的其他终端对于所述互动数据的互动反馈数据，并将所述互动反馈数据显示到所述目标显示窗口。
  14. 如权利要求9所述的计算机设备，所述计算机可读指令被所述处理器执行时还实现以下步骤：
    将所述互动反馈数据通过所述互动连接通道发送至所述投屏终端，其中，所述互动数据类型包括文字、图片、弹幕、打赏、礼物中的至少一种。
  15. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机可读指令，所述计算机可读指令可被至少一个处理器执行，以使所述至少一个处理器执行以下步骤：
    接收投屏终端的投屏请求，建立与所述投屏终端之间的音视频数据连接通道；
    通过所述音视频数据连接通道获取音视频数据并显示到目标显示窗口；
    接收所述投屏终端的互动请求，获取所述互动请求中的互动数据类型；
    根据互动数据类型建立与所述投屏终端之间的互动数据连接通道；
    通过所述互动数据连接通道接收所述投屏终端的互动数据，将所述互动数据显示到所述目标显示窗口。
  16. 如权利要求15所述的计算机可读存储介质，所述通过所述音视频数据连接通道获取音视频数据包括：
    获取所述投屏请求中的所述音视频数据的资源地址；
    透过所述音视频数据连接通道与所述资源地址进行连接，并访问所述音视频数据。
  17. 如权利要求15所述的计算机可读存储介质，所述根据互动数据类型建立与所述投屏终端之间的互动数据连接通道包括：
    根据所述互动数据类型从预设的互动数据连接通道信息表中查询对应的互动数据连接通道类型及建立规则；
    根据所述建立规则建立所述互动数据连接通道类型的互动数据连接通道。
  18. 如权利要求17所述的计算机可读存储介质，所述建立规则包括通过预设协议建立数据通信链路；所述互动数据连接通道包括支持至少一种互动数据类型的互动数据进行传输的通信链路。
  19. 如权利要求15所述的计算机可读存储介质，所述计算机可读指令可被至少一个处理器执行，还使所述至少一个处理器执行以下步骤：
    将所述互动数据发送至互动服务器；
    接收由所述互动服务器返回的其他终端对于所述互动数据的互动反馈数据，并将所述互动反馈数据显示到所述目标显示窗口。
  20. 如权利要求15所述的计算机可读存储介质，所述计算机可读指令可被至少一个处理器执行，还使所述至少一个处理器执行以下步骤：
    将所述互动反馈数据通过所述互动连接通道发送至所述投屏终端，其中，所述互动数据类型包括文字、图片、弹幕、打赏、礼物中的至少一种。
PCT/CN2020/141032 2020-05-06 2020-12-29 多屏互动显示方法及装置 WO2021223453A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/923,226 US11910072B2 (en) 2020-05-06 2020-12-29 Multi-screen interactive display method and apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010374621.1 2020-05-06
CN202010374621.1A CN111629240B (zh) 2020-05-06 2020-05-06 多屏互动显示方法及装置

Publications (1)

Publication Number Publication Date
WO2021223453A1 true WO2021223453A1 (zh) 2021-11-11

Family

ID=72261130

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/141032 WO2021223453A1 (zh) 2020-05-06 2020-12-29 多屏互动显示方法及装置

Country Status (3)

Country Link
US (1) US11910072B2 (zh)
CN (1) CN111629240B (zh)
WO (1) WO2021223453A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114296848A (zh) * 2021-12-23 2022-04-08 深圳市宝视达光电有限公司 会议一体机、屏幕切分显示方法以及存储装置
CN114567810A (zh) * 2022-02-28 2022-05-31 深圳创维－Rgb电子有限公司 投屏交互方法、装置、投屏器及存储介质
CN114827690A (zh) * 2022-03-30 2022-07-29 北京奇艺世纪科技有限公司 一种网络资源显示方法、装置及系统
CN115022694A (zh) * 2022-06-27 2022-09-06 北京奇艺世纪科技有限公司 一种投屏时间统计方法、装置、系统、后台服务器及介质
CN115174537A (zh) * 2022-06-30 2022-10-11 重庆长安汽车股份有限公司 一种车机交互的双屏互动方法及系统
CN115379279A (zh) * 2022-07-22 2022-11-22 深圳市酷开网络科技股份有限公司 多屏联动的交互方法及装置、系统、存储介质、电子设备

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111629240B (zh) * 2020-05-06 2021-08-10 上海幻电信息科技有限公司 多屏互动显示方法及装置
CN114173184B (zh) * 2020-09-10 2024-10-18 华为终端有限公司 投屏方法和电子设备
CN112732212A (zh) * 2020-12-31 2021-04-30 咪咕音乐有限公司 显示方法、电子设备及存储介质
CN114827742B (zh) * 2021-01-29 2024-04-30 抖音视界有限公司 直播互动方法、装置、计算机设备及计算机可读介质
CN113596530A (zh) * 2021-07-23 2021-11-02 北京沃东天骏信息技术有限公司 用于流转数据的方法及装置

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130179605A1 (en) * 2012-01-06 2013-07-11 Qualcomm Incorporated Bi-directional tunneling via user input back channel for wireless displays
CN105828171A (zh) * 2016-03-14 2016-08-03 乐视网信息技术（北京）股份有限公司 智能互联方法和智能终端
CN106792055A (zh) * 2016-12-28 2017-05-31 福建星网视易信息系统有限公司 实时投屏方法、设备及系统
CN108616764A (zh) * 2016-12-09 2018-10-02 中兴通讯股份有限公司 一种控制方法及设备
CN108668027A (zh) * 2018-05-22 2018-10-16 武汉卡比特信息有限公司 手机互联时分屏投射到计算机类终端并反向控制的系统及方法
CN109996097A (zh) * 2019-03-12 2019-07-09 广州虎牙信息科技有限公司 一种投屏方法、系统及存储装置
CN111309279A (zh) * 2020-05-06 2020-06-19 上海幻电信息科技有限公司 多屏互动显示方法及装置
CN111629240A (zh) * 2020-05-06 2020-09-04 上海幻电信息科技有限公司 多屏互动显示方法及装置

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8281339B1 (en) * 2004-01-12 2012-10-02 United Video Properties, Inc. Customizable flip and browse overlays in an interactive television system
US20100263005A1 (en) * 2009-04-08 2010-10-14 Eric Foster White Method and system for egnaging interactive web content
CN103581704B (zh) * 2012-07-31 2017-07-07 华为技术有限公司 实现多屏互动的方法和装置
CN103067383B (zh) * 2012-12-27 2016-06-29 华为终端有限公司 无线上网设备和多媒体通信方法
CN105245801A (zh) * 2015-09-24 2016-01-13 天脉聚源（北京）科技有限公司 一种传送电视互动系统互动信号的方法
CN105791291B (zh) * 2016-03-02 2019-09-03 腾讯科技（深圳）有限公司 网络应用的显示控制方法、显示中实时更新的方法和装置
US10142699B2 (en) * 2016-03-18 2018-11-27 Google Llc Systems and methods for providing interactive content
US10110968B2 (en) * 2016-04-19 2018-10-23 Google Llc Methods, systems and media for interacting with content using a second screen device
CN108769814B (zh) * 2018-06-01 2022-02-01 腾讯科技（深圳）有限公司 视频互动方法、装置、终端及可读存储介质
CN108829482B (zh) * 2018-06-20 2022-04-05 广州视源电子科技股份有限公司 数据传输协作控制系统、方法、存储介质及交互设备
CN109474670A (zh) * 2018-10-22 2019-03-15 中国人民解放军战略支援部队航天工程大学 一种信息交互方法及系统
CN110109636B (zh) * 2019-04-28 2022-04-05 华为技术有限公司 投屏方法、电子设备以及系统
CN110392226A (zh) * 2019-06-19 2019-10-29 视联动力信息技术股份有限公司 一种直播实现方法和装置
CN110597474B (zh) * 2019-09-12 2022-03-25 联想(北京)有限公司 信息处理方法及电子设备
CN110602805B (zh) * 2019-09-30 2021-06-15 联想(北京)有限公司 信息处理方法、第一电子设备和计算机系统
CN110620953A (zh) * 2019-10-08 2019-12-27 赛诺贝斯（北京）营销技术股份有限公司 一种多屏互动方法、装置及系统
CN110913013B (zh) * 2019-12-06 2022-07-29 贵州电网有限责任公司 多屏互动方法及系统

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130179605A1 (en) * 2012-01-06 2013-07-11 Qualcomm Incorporated Bi-directional tunneling via user input back channel for wireless displays
CN105828171A (zh) * 2016-03-14 2016-08-03 乐视网信息技术（北京）股份有限公司 智能互联方法和智能终端
CN108616764A (zh) * 2016-12-09 2018-10-02 中兴通讯股份有限公司 一种控制方法及设备
CN106792055A (zh) * 2016-12-28 2017-05-31 福建星网视易信息系统有限公司 实时投屏方法、设备及系统
CN108668027A (zh) * 2018-05-22 2018-10-16 武汉卡比特信息有限公司 手机互联时分屏投射到计算机类终端并反向控制的系统及方法
CN109996097A (zh) * 2019-03-12 2019-07-09 广州虎牙信息科技有限公司 一种投屏方法、系统及存储装置
CN111309279A (zh) * 2020-05-06 2020-06-19 上海幻电信息科技有限公司 多屏互动显示方法及装置
CN111629240A (zh) * 2020-05-06 2020-09-04 上海幻电信息科技有限公司 多屏互动显示方法及装置

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114296848A (zh) * 2021-12-23 2022-04-08 深圳市宝视达光电有限公司 会议一体机、屏幕切分显示方法以及存储装置
CN114296848B (zh) * 2021-12-23 2024-03-19 深圳市宝视达光电有限公司 会议一体机、屏幕切分显示方法以及存储装置
CN114567810A (zh) * 2022-02-28 2022-05-31 深圳创维－Rgb电子有限公司 投屏交互方法、装置、投屏器及存储介质
CN114827690A (zh) * 2022-03-30 2022-07-29 北京奇艺世纪科技有限公司 一种网络资源显示方法、装置及系统
CN114827690B (zh) * 2022-03-30 2023-07-25 北京奇艺世纪科技有限公司 一种网络资源显示方法、装置及系统
CN115022694A (zh) * 2022-06-27 2022-09-06 北京奇艺世纪科技有限公司 一种投屏时间统计方法、装置、系统、后台服务器及介质
CN115022694B (zh) * 2022-06-27 2023-10-27 北京奇艺世纪科技有限公司 一种投屏时间统计方法、装置、系统、后台服务器及介质
CN115174537A (zh) * 2022-06-30 2022-10-11 重庆长安汽车股份有限公司 一种车机交互的双屏互动方法及系统
CN115174537B (zh) * 2022-06-30 2023-07-21 重庆长安汽车股份有限公司 一种车机交互的双屏互动方法及系统
CN115379279A (zh) * 2022-07-22 2022-11-22 深圳市酷开网络科技股份有限公司 多屏联动的交互方法及装置、系统、存储介质、电子设备

Also Published As

Publication number Publication date
US20230188813A1 (en) 2023-06-15
US11910072B2 (en) 2024-02-20
CN111629240B (zh) 2021-08-10
CN111629240A (zh) 2020-09-04

Similar Documents

Publication Publication Date Title
WO2021223453A1 (zh) 多屏互动显示方法及装置
US11695979B2 (en) Method and device of displaying multi-screen interactive
US9686329B2 (en) Method and apparatus for displaying webcast rooms
WO2020094034A1 (zh) 用于终端的无线网络连接方法
EP4262214A1 (en) Screen projection method and apparatus, and electronic device and storage medium
CN112073754B (zh) 一种云游戏投屏方法、装置、计算机设备、计算机可读存储介质及云游戏投屏交互系统
WO2020094035A1 (zh) 用于终端的无线网络连接方法
US9686506B2 (en) Method, apparatus, system, and storage medium for video call and video call control
CN104539977A (zh) 直播预览方法及装置
CN109586929B (zh) 会议内容的传输方法及装置、电子设备、存储介质
US20170171585A1 (en) Method and Electronic Device for Recording Live Streaming Media
US12041688B2 (en) Screen projection method and system
CN112261467A (zh) 投屏内容展示方法、装置、电子设备和可读存储介质
CN111405042A (zh) 一种电子设备的发现方法、装置、计算机可读存储介质和电子装置
CN112312222A (zh) 视频发送方法、装置和电子设备
WO2015035957A1 (zh) 一种视频资源获取方法、后台服务器、视频客户端及系统
EP3048796A1 (en) Information system, information delivery method and iptv system based on multi-screen interaction
CN103365061A (zh) 云投影系统
CN109348298B (zh) 信息推送、播放多媒体数据流的方法和设备
CN103365062A (zh) 云投影方法
CN107273125B (zh) 一种远程切换屏幕模式的方法及系统
CN110620761B (zh) 一种实现多人虚拟互动的方法与装置
CN114222176A (zh) 一种同步投屏方法、设备及存储介质
US9232281B2 (en) Communication management apparatus, terminal, communication management system, communication management method, program, and information storage medium
CN114942812B (zh) 多媒体信息共享方法、装置和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20934637

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20934637

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17-05-2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20934637

Country of ref document: EP

Kind code of ref document: A1