WO2018095174A1 - Control method, device, and terminal apparatus for synthesizing video stream of live streaming room - Google Patents

Control method, device, and terminal apparatus for synthesizing video stream of live streaming room Download PDF

Info

Publication number
WO2018095174A1
WO2018095174A1 PCT/CN2017/107317 CN2017107317W WO2018095174A1 WO 2018095174 A1 WO2018095174 A1 WO 2018095174A1 CN 2017107317 W CN2017107317 W CN 2017107317W WO 2018095174 A1 WO2018095174 A1 WO 2018095174A1
Authority
WO
WIPO (PCT)
Prior art keywords
video stream
video
stream
layout
user
Prior art date
Application number
PCT/CN2017/107317
Other languages
French (fr)
Chinese (zh)
Inventor
于川
余蒙
徐光兴
吴昊
苏庆辉
陆锦铃
郭业翔
Original Assignee
广州华多网络科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州华多网络科技有限公司 filed Critical 广州华多网络科技有限公司
Publication of WO2018095174A1 publication Critical patent/WO2018095174A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/475—End-user interface for inputting end-user data, e.g. personal identification number [PIN], preference data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting

Definitions

  • the present invention relates to the field of network live broadcast technology, and in particular, to a video stream synthesis control method, device and terminal device between live broadcasts.
  • the current live broadcast platform includes a live broadcast platform for the mobile terminal and a live broadcast platform for the PC end.
  • the user needs to simultaneously broadcast content of multiple mobile terminals or simultaneously broadcast content of multiple PCs or simultaneously broadcast content of the mobile terminal and the PC end, for example, the user starts to broadcast live on the PC, and then wants to temporarily transfer the live broadcast scene.
  • the anchor user is often required to re-establish a live broadcast room on the mobile terminal to open the outdoor live broadcast, and then the viewing user in the original live broadcast room needs to re-enter the new live broadcast room. After watching the outdoor broadcast of the anchor, the outdoor broadcast is temporary.
  • the anchor After the outdoor broadcast stops, the anchor returns to the PC to broadcast live, and the viewing user has to re-enter the live broadcast of the host's PC.
  • the operation process of the next anchor user and the viewing user is cumbersome, and it is impossible to bring a good experience to the anchor user and the viewing user, and reduce the reputation of the live broadcast platform, resulting in user loss.
  • the anchor user wants to invite friends and viewers to broadcast together. Under the current technology, only the friends or the viewers are also the anchor users can broadcast together.
  • the primary object of the present invention is to provide a method and apparatus for controlling video stream synthesis between live broadcasts.
  • Another object of the present invention is to provide a terminal device that implements and implements the above-described live video stream synthesis control method.
  • the present invention provides a method for controlling video stream synthesis between live broadcasts, including the following steps:
  • the first, second, and third video streams each include an image stream and an audio stream
  • the third video stream includes an image stream of at least one of the first video stream and the second video stream, and further includes the two At least one of the audio streams.
  • the selected instruction includes a selected instruction that the at least one video control is selected by the anchor user, and a selected instruction that selects at least one video control according to a preset rule.
  • the selected instruction further includes a selected instruction selected by the anchor user as one of a plurality of preset layout layouts as the specified layout of the third video stream.
  • the layout layout includes an image position of the image streams of the first and second video streams in the synthesized third video stream, an image size, and a layout layout of the image tilt angle.
  • the method further includes the steps of:
  • the method further includes the steps of:
  • the audio stream of the second video stream is turned on/off or the size of the audio stream of the second video stream is adjusted in response to an operation instruction of the audio stream acting on the second video stream.
  • the process of receiving the second video stream and the process of collecting the local first video stream and the process of synthesizing the third video stream work in parallel with the process of uploading the third video stream.
  • the present invention further provides a live stream video stream synthesis control apparatus, including:
  • Calling module for displaying one or more video controls corresponding to the number of second video streams
  • a synthesis module responsive to a selected instruction acting on at least one of the video controls, receiving a second video stream corresponding to the selected video control, and the second video stream and the locally acquired A video stream is synthesized into a third video stream;
  • the uploading module is configured to upload the third video stream to the server, so that the server pushes the third video stream to each user in the live broadcast room.
  • the present invention further provides a terminal device for performing and implementing the above-mentioned live video stream synthesis control method, comprising a processor and a memory, wherein the processor is configured to invoke to execute the live broadcast video stored in the memory in a program form.
  • Flow synthesis control method The steps of any of the methods described.
  • the present invention has the following advantages:
  • the present invention configures the anchor terminal as a server terminal for receiving the second video stream directly sent by the second terminal to the anchor terminal, and then the allcast video stream is synthesized and pushed by the anchor terminal to the video stream of the live broadcast room, and the live broadcast is performed.
  • the video stream seen by the viewer user is consistent with the video stream on the anchor terminal, ensuring the simultaneity of the video stream;
  • the long link is used to maintain the communication link between the anchor terminal and the second terminal.
  • the reception is selected.
  • the second video stream, and the process of selecting to connect any one of the second video streams can be selected in a random manner and displayed in the live broadcast screen to enhance the interaction effect between the anchor and the viewer user;
  • the anchor can adjust the screen layout and audio effects of the synthesized video stream to better highlight the live content and create a better live broadcast effect.
  • FIG. 1 is a schematic flow chart of an embodiment of a method for controlling video stream synthesis between live broadcasts according to the present invention
  • FIG. 2 is a schematic diagram of a live broadcast interface after the multi-source live broadcast mode is enabled on the anchor terminal of the present invention
  • FIG. 3 is a schematic layout diagram of a third video stream after selecting two second video streams according to the present invention.
  • FIG. 4 is a schematic diagram of an embodiment of a video stream synthesis control device between broadcasts according to the present invention.
  • FIG. 5 is a schematic diagram of a system for implementing a video stream synthesis control method between broadcasts according to the present invention.
  • terminal and terminal device used herein include both a wireless signal receiver device, a device having only a wireless signal receiver without a transmitting capability, and a receiving and transmitting hardware.
  • Such devices may include cellular or other communication devices having a single line display or a multi-line display or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service), which may combine voice, data processing, facsimile and / or data communications capabilities; PDA (personal digital Assistant, personal digital assistants), which may include network access to a radio frequency receiver, a pager, Internet / web browser, notepad, calendar and / or GPS (Global Positioning System (Global Positioning System) receiver; conventional laptop and/or palmtop computer or other device having a conventional laptop and/or palmtop computer or other device that includes and/or includes a radio frequency receiver.
  • GPS Global Positioning System
  • terminal may be portable, transportable, installed in a vehicle (aviation, sea and/or land), or adapted and/or configured to operate locally, and/or Run in any other location on the Earth and/or space in a distributed form.
  • the "terminal” and “terminal device” used herein may also be a communication terminal, an internet terminal, a music/video playing terminal, and may be, for example, a PDA, a MID (Mobile Internet Device), and/or have a music/video playback.
  • Functional mobile phones can also be smart TVs, set-top boxes and other devices.
  • the remote network device used herein includes, but is not limited to, a computer, a network host, a single network server, a plurality of network server sets, or a cloud composed of multiple servers.
  • the cloud is composed of a large number of computers or network servers based on Cloud Computing, which is a kind of distributed computing, a super virtual computer composed of a group of loosely coupled computers.
  • the communication between the remote network device, the terminal device and the WNS server can be implemented by any communication method, including but not limited to, mobile communication based on 3GPP, LTE, WIMAX, TCP/IP, UDP protocol. Computer network communication and short-range wireless transmission based on Bluetooth and infrared transmission standards.
  • the user interface/operation interface of the present invention generally refers to a display interface that can be used to send control instructions to the smart terminal, for example, an option in the setting page of the Andro i d system (or a button, added by the application, the same below), may also be an option in the notification bar or interactive page that is called from the desktop, or may be an option in a page constructed by an active component of the application. . While some exemplary embodiments of the invention have been shown in the foregoing, the embodiments of the invention may The scope is defined by the claims and their equivalents.
  • the live broadcast room includes the following meanings: 1 a virtual space (or virtual room) created based on the webcast platform.
  • the live broadcast room is generally created by the anchor client and connected to multiple viewing clients, that is, live broadcast.
  • the anchor and the plurality of viewers are included in the room, and the viewing client located in the virtual space can watch the live content of the anchor client, and the user of the anchor client and the user who views the client, the user who views the client, and the user who views the client.
  • 2 an instant messaging platform that aggregates users together in groups, such as a video conferencing system, where users log in to the client to enter the group, and the user is in the group.
  • the group exists as a member of the group.
  • the same group contains multiple group members.
  • the user can join or leave the group arbitrarily. Within the group, various interactions such as text, voice, and video can be performed.
  • FIG. 1 is a schematic flowchart of an embodiment of a method for controlling video stream synthesis in a live broadcast according to the present invention, including the following steps:
  • the first video stream is a video stream collected by an anchor user terminal that starts a live broadcast by an anchor identity
  • the second video stream is collected by a second user terminal connected to the anchor terminal.
  • Video stream is a video stream collected by an anchor user terminal that starts a live broadcast by an anchor identity
  • Step S100 Display one or more video controls corresponding to the number of second video streams.
  • Multi-source live broadcast refers to a live broadcast mode in which a plurality of video streams of multiple sources exist simultaneously in one live broadcast screen, and the video streams of the multiple sources may be multiple video streams collected by the anchor user using multiple photographing devices or the host user invites other
  • the server terminal is configured to receive the second video stream sent by the second terminal, and before the anchor user confirms to receive the second video stream, the second terminal and the anchor terminal maintain a communication link connection between the two parties in a long connection manner.
  • the second terminal needs to send a heartbeat data packet to the anchor terminal at intervals to maintain the long link, and after the anchor user confirms receiving the second video stream, the second terminal passes the The communication link sends the second video stream to the anchor terminal.
  • one or more second terminals that maintain a long connection with the anchor terminal are displayed on the user interface of the anchor terminal in the form of a video control, and each video control corresponds to a second video.
  • a selection entry for the stream, and also included in the video control for distinguishing The feature information of the second video stream can be triggered to receive the corresponding second video stream by selecting a video control.
  • the feature information is a user name of the second terminal user on the live broadcast platform.
  • FIG. 2 is a schematic diagram of a live broadcast interface after the multi-source live broadcast mode is enabled on the anchor terminal.
  • FIG. 2 shows that four second terminals apply for connection with the anchor terminal for multi-source live broadcast, and four video controls are arranged above the live broadcast interface. Displaying a second video stream of the 4 second terminals that are connected to the anchor terminal, and adding feature information for distinguishing different second video streams, that is, the user name of the second terminal user on the live broadcast platform As shown in Figure 2, "The first demon male of Huya”, “Old Xing girl sds", "The old civet old driver” and "Different every day.".
  • Step S200 Receive a second video stream corresponding to the selected video control in response to the selected instruction acting on at least one of the video controls, and first capture the second video stream with the local device in a specified layout.
  • the video stream is synthesized into a third video stream.
  • the video control displayed on the anchor terminal user interface triggers a corresponding selected command when a selected operation occurs on at least one of the video controls, and the anchor terminal starts receiving in response to the selected command
  • the video control corresponds to the second video stream, and combines the second video stream with the locally collected first video stream into a third video stream in a specified layout.
  • An embodiment of the selected instruction is that the selected instruction is a selected instruction for selecting at least one video control by an anchor user, wherein the anchor user holds the selection of any one or more second terminal users. Controlling the video control, the anchor user may select one or more video controls corresponding to the second terminal user according to the preference of the second terminal user to receive the second video stream corresponding to the video control;
  • Another implementation of the instruction in order to enhance the activity of the anchor and the user, activate the atmosphere between the live broadcasts, and let more users participate, the selection is designated as selecting a selected instruction of the at least one video control according to a preset rule, preferably The preset rule is that one or more video controls are randomly selected from a plurality of video controls corresponding to the plurality of connected second video streams to receive the second video stream corresponding to the video control, and randomly selected
  • the process also appears in the third video stream, which can enhance the master when there is a limit on the number of second video streams that maintain a long connection. With interaction effects between users.
  • the selected instruction further includes a selected instruction selected by the anchor user as one of a plurality of preset layout layouts as the specified layout of the third video stream.
  • the anchor user needs to select the layout layout of the third video stream, wherein the anchor terminal presets the layout layout of the plurality of third video streams, including the connection 1 , 2, 3 second video streams a layout layout of the time, and the layout layout includes an image position of the image streams of the first and second video streams in the synthesized third video stream, an image size, and a layout layout of the image tilt angle, as shown in FIG.
  • a layout layout of the third video stream after connecting two second video streams ("the first demon male" and "the old sds").
  • the anchor user may further modify the layout layout of the third video stream, and the anchor terminal response acts on the Editing instructions of the layout of the third video stream on the user interface, changing the image layout layout effect of the third video stream, wherein the editing of the layout layout for the third video stream includes image streams for the first and second video streams The layout position of the image position, the image size, and the tilt angle of the image in the synthesized third video stream, as shown in FIG.
  • the anchor user selects one of the second video streams located in the upper right corner of the live video screen, and the second video stream
  • the screen is highlighted by a frame line, and the anchor user can modify the image size of the graphics stream of the second video stream by dragging the frame line, and counting from left to right by clicking below the second video stream screen.
  • the second button is to modify the image tilt angle of the graphics stream of the second video stream.
  • the embodiment further includes the step of: turning on/off the audio stream of the second video stream or adjusting the size of the audio stream of the second video stream in response to an operation instruction of the audio stream acting on the second video stream.
  • the anchor user may also adjust the audio stream of the second video stream as needed, including turning on/off the audio stream of the second video stream and adjusting the size of the audio stream of the second video stream, generally, the image frame of the second video stream.
  • the operation button of the audio stream of the second video stream is provided, and the third button from the left to the right of the screen of the second video stream in the upper right corner of the live video screen shown in FIG. 3 is used, and the anchor user can operate the button. Button to turn on/off the audio stream of the second video stream and adjust the size of the audio stream of the second video stream.
  • Step S300 Upload the third video stream to a server, so that the server pushes the third video stream to each user in the live broadcast room.
  • the anchor terminal After synthesizing the third video stream, the anchor terminal uploads the third video stream to the server of the live broadcast platform, so that the server pushes the third video stream to each user in the live broadcast room.
  • the The process of receiving the second video stream by the live broadcast terminal and the process of collecting the local first video stream and the process of synthesizing the third video stream work in parallel with the process of uploading the third video stream, and the parallel work refers to completion at the same time or at the same time interval.
  • the parallel operation means that the anchor terminal can synchronously/synchronize the first video stream when receiving the second direct stream, and simultaneously/synchronize the collected first video stream and the The received second video stream is merged into a third video stream, during which time, the modification of the layout layout of the third video stream, the operation of the audio stream of the second video stream, and the simultaneous/synchronization of the third video are also included.
  • the stream is uploaded to the server to ensure the simultaneous/synchronization of the live video stream.
  • FIG. 2 is a schematic diagram of an embodiment of a live video stream synthesis control apparatus according to the present invention, including:
  • the calling module 100 is configured to display one or more video controls corresponding to the number of the second video streams.
  • the anchor user starts the multi-source live broadcast mode, and the second terminal applies for connection with the anchor terminal to perform multi-source live broadcast.
  • the anchor terminal at this time is configured as a server terminal, and is configured to receive the second terminal.
  • the second terminal and the anchor terminal maintain a communication link connection between the two parties in a long connection manner, and during the long connection hold period, if there is no video stream data transmission, the first The second terminal sends a heartbeat data packet to the anchor terminal at intervals to maintain the long link.
  • the second terminal sends the second video stream to the anchor terminal through the communication link.
  • the calling module 100 displays one or more second terminals (ie, the second video stream) that are connected to the anchor terminal in a long form in the form of a video control on the user interface of the anchor terminal, and also in the video control.
  • the feature information is used to distinguish the plurality of second video streams.
  • the feature information is a user of the second terminal user on the live broadcast platform. .
  • the synthesizing module 200 is configured to receive a second video stream corresponding to the selected video control in response to the selected instruction acting on the at least one of the video controls, and to locally capture the second video stream with a specified layout
  • the first video stream is synthesized into a third video stream.
  • the calling module 100 displays the video control on the anchor terminal user interface, when a selected operation occurs on at least one of the video controls, a corresponding selected instruction is triggered, and the synthesizing module 200 is responsive to the selected instruction.
  • the synthesizing module 200 starts receiving a second video stream corresponding to the selected video control, and synthesizing the second video stream with the locally collected first video stream into a third video stream in a specified layout.
  • Uploading module 300 for uploading the third video stream to a server, so that the service The device pushes the third video stream to each user in the live room.
  • the uploading module 300 uploads the third video stream to the server of the live broadcast platform, so that the server pushes the third video stream to each user in the live broadcast room, in the implementation.
  • the process of receiving the second video stream by the synthesizing module 200 and the process of collecting the local first video stream and the process of synthesizing the third video stream work in parallel with the process of uploading the third video stream by the uploading module 300.
  • the parallel operation means that the synthesizing module 200 can synchronously/synchronize the first video stream when receiving the second direct stream, and simultaneously/synchronize the collected first video stream and the received second video stream into a first
  • the three video streams, during which the modification of the layout of the third video stream, the operation of the audio stream of the second video stream, and the uploading module 300 simultaneously/synchronizing the third video stream to the server are ensured. Simultaneous/synchronization of live video streams.
  • the present invention further provides a terminal device, including a memory, a processor, configured to store candidate intermediate data and result data generated in the implementation of the foregoing method, where the processor is used to execute execution in a program form. Any one of the steps of the live stream video stream synthesis control method of the memory.
  • FIG. 5 is a schematic diagram of a system for implementing a video stream synthesis control method according to the present invention.
  • the present invention configures the anchor terminal as a server terminal for receiving the second video stream directly sent by the second terminal to the anchor terminal, and then the allcast video stream is synthesized and pushed by the anchor terminal to the video stream in the live broadcast room, and the audience in the live broadcast room
  • the video stream seen by the user is consistent with the video stream on the anchor terminal, ensuring the simultaneity of the video stream; at the same time, the long link is used to maintain the communication link between the anchor terminal and the second terminal, and for the second video stream of multiple sources,
  • the process of receiving the selected second video stream and selecting to connect any one of the second video streams may be selected in a random manner and displayed on the live broadcast screen to enhance the anchor.
  • the interactive effect with the viewer user in addition, the anchor can adjust the screen layout and audio effects of the synthesized video stream to better highlight the live content and create a better live broadcast effect.

Abstract

The present invention relates to the technical field of network live streaming, and particularly relates to a control method, device, and terminal apparatus for synthesizing a video stream of a live streaming room. The method comprises steps of: displaying one or more video controls corresponding to the number of second video streams; receiving, in response to at least one selection instruction acting on the video controls, a second video stream corresponding to a selected video control, and synthesizing, using a specified layout, said second video stream and a locally captured first video stream into a third video stream; and uploading the third video stream to a server, such that the server pushes the third video stream to each user in the live streaming room. By configuring a broadcaster terminal as a server terminal, the present invention reduces bandwidth usage, and lowers a workload of a live streaming platform server. In addition, by selecting, in different manners, a connected second video stream in a multi-source live streaming mode, the present invention improves interaction between a broadcaster and a user to create a lively atmosphere in a live streaming room, and enhances presentation of live streaming content in a multi-source live streaming mode to audience and users in the live streaming room.

Description

直播间视频流合成控制方法、装置及终端设备Video stream synthesis control method, device and terminal device in live broadcast room 【技术领域】[Technical Field]
本发明涉及网络直播技术领域，具体涉及一种直播间视频流合成控制方法、装置及终端设备。The present invention relates to the field of network live broadcast technology, and in particular, to a video stream synthesis control method, device and terminal device between live broadcasts.
【背景技术】【Background technique】
随着互联网技术及智能移动终端设备的发展，各种互联网产品给人们的工作、生活带来了很多便利与娱乐，近年来，各类用于视频直播的直播平台层出不穷，视频直播给人们带来更实时的社交体验。目前的直播平台包括了用于移动端的直播平台和用于PC端的直播平台。With the development of Internet technology and intelligent mobile terminal devices, various Internet products have brought a lot of convenience and entertainment to people's work and life. In recent years, various live broadcast platforms for live video broadcasts have emerged in an endless stream. A more real-time social experience. The current live broadcast platform includes a live broadcast platform for the mobile terminal and a live broadcast platform for the PC end.
由于视频直播需求的多样性，用户需要同时直播多个移动端的内容或同时直播多个PC端的内容或同时直播移动端跟PC端的内容，例如用户开始在PC端上直播，然后想暂时转移直播场景至户外，利用移动端直播户外此时发生的情况，在现有技术中，往往需要主播用户在移动端重新建立一个直播间开启户外直播，然后原来直播间的观看用户需要重新进入新的直播间后才能看到主播进行的户外直播，由于户外直播是暂时性的，在户外直播停止后，主播重新回到PC端直播，而观看用户又得重新进入该主播的PC端的直播间，这种方式下主播用户跟观看用户的操作过程繁琐，无法给主播用户跟观看用户带来良好体验，降低直播平台口碑，造成用户流失。另外，例如主播用户想要邀请好友、观众一起直播，在现在技术下，只有好友或观众亦是主播用户才可以进行一起直播。Due to the diversity of video live broadcast requirements, the user needs to simultaneously broadcast content of multiple mobile terminals or simultaneously broadcast content of multiple PCs or simultaneously broadcast content of the mobile terminal and the PC end, for example, the user starts to broadcast live on the PC, and then wants to temporarily transfer the live broadcast scene. To the outdoors, using the mobile terminal to broadcast the outdoor situation at this time, in the prior art, the anchor user is often required to re-establish a live broadcast room on the mobile terminal to open the outdoor live broadcast, and then the viewing user in the original live broadcast room needs to re-enter the new live broadcast room. After watching the outdoor broadcast of the anchor, the outdoor broadcast is temporary. After the outdoor broadcast stops, the anchor returns to the PC to broadcast live, and the viewing user has to re-enter the live broadcast of the host's PC. The operation process of the next anchor user and the viewing user is cumbersome, and it is impossible to bring a good experience to the anchor user and the viewing user, and reduce the reputation of the live broadcast platform, resulting in user loss. In addition, for example, the anchor user wants to invite friends and viewers to broadcast together. Under the current technology, only the friends or the viewers are also the anchor users can broadcast together.
所以，如何在多个终端同时直播的多源直播模式下更好地对多个来源的视频流进行管控，如何在多源直播模式下实现主播跟观众更好的互动效果，并且如何对视频流的合成进行更好地控制以提升直播的质量跟内容是目前网络视频直播领域中亟需解决的问题。Therefore, how to better control video streams of multiple sources in a multi-source live broadcast mode in which multiple terminals are simultaneously broadcasted, how to achieve better interaction between the anchor and the viewer in the multi-source live broadcast mode, and how to perform video streaming The better control of the synthesis to improve the quality and content of the live broadcast is an urgent problem in the field of online video live broadcasting.
【发明内容】[Summary of the Invention]
本发明的首要目的在于提供一种直播间视频流合成控制方法及装置。 The primary object of the present invention is to provide a method and apparatus for controlling video stream synthesis between live broadcasts.
本发明的另一目的在于提供一种实行、实现上述直播间视频流合成控制方法的终端设备。Another object of the present invention is to provide a terminal device that implements and implements the above-described live video stream synthesis control method.
为实现该目的，本发明采用如下技术方案：To achieve this, the present invention adopts the following technical solutions:
第一方面，本发明提供一种直播间视频流合成控制方法，包括如下步骤：In a first aspect, the present invention provides a method for controlling video stream synthesis between live broadcasts, including the following steps:
显示与第二视频流个数相对应的一个或多个视频控件；Displaying one or more video controls corresponding to the number of second video streams;
响应作用于所述视频控件的至少一个的选定指令，接收被选定的视频控件相对应的第二视频流，并以指定布局将该第二视频流与本地采集的第一视频流合成为第三视频流；Responding to a selected instruction acting on at least one of the video controls, receiving a second video stream corresponding to the selected video control, and synthesizing the second video stream with the locally acquired first video stream in a specified layout as Third video stream;
将所述第三视频流上传到服务器，以使得所述服务器向直播间的每个用户推送所述第三视频流。Uploading the third video stream to a server, so that the server pushes the third video stream to each user in the live room.
具体的，所述第一、第二、第三视频流均包括图像流和音频流，所述第三视频流包括第一视频流、第二视频流至少之一的图像流，还包括该两者至少之一的音频流。Specifically, the first, second, and third video streams each include an image stream and an audio stream, and the third video stream includes an image stream of at least one of the first video stream and the second video stream, and further includes the two At least one of the audio streams.
进一步的，所述选定指令包括由主播用户选择至少一个视频控件的选定指令，和根据预设规则选择至少一个视频控件的选定指令。Further, the selected instruction includes a selected instruction that the at least one video control is selected by the anchor user, and a selected instruction that selects at least one video control according to a preset rule.
更进一步的，所述选定指令还包括由主播用户选用预设的多个排版布局之一作为第三视频流的指定布局的选定指令。Further, the selected instruction further includes a selected instruction selected by the anchor user as one of a plurality of preset layout layouts as the specified layout of the third video stream.
具体的，所述排版布局包括第一、第二视频流的图像流在合成的第三视频流中的图像位置、图像大小、图像倾斜角度的排版布局。Specifically, the layout layout includes an image position of the image streams of the first and second video streams in the synthesized third video stream, an image size, and a layout layout of the image tilt angle.
进一步的，所述的方法还包括步骤：Further, the method further includes the steps of:
响应作用于所述第三视频流在用户界面上的排版布局的编辑指令，改变第三视频流的图像排版布局效果。And responding to an editing instruction of the layout layout of the third video stream on the user interface, changing an image layout effect of the third video stream.
更进一步的，所述的方法还包括步骤：Further, the method further includes the steps of:
响应作用于第二视频流的音频流的操作指令，打开/关闭第二视频流的音频流或调整第二视频流的音频流的大小。The audio stream of the second video stream is turned on/off or the size of the audio stream of the second video stream is adjusted in response to an operation instruction of the audio stream acting on the second video stream.
优选的，所述接收第二视频流的过程与采集本地的第一视频流的过程与合成第三视频流的过程与上传第三视频流的过程并行工作。Preferably, the process of receiving the second video stream and the process of collecting the local first video stream and the process of synthesizing the third video stream work in parallel with the process of uploading the third video stream.
第二方面，本发明还提供一种直播间视频流合成控制装置，包括：In a second aspect, the present invention further provides a live stream video stream synthesis control apparatus, including:
调用模块：用于显示与第二视频流个数相对应的一个或多个视频控件； Calling module: for displaying one or more video controls corresponding to the number of second video streams;
合成模块：用于响应作用于所述视频控件的至少一个的选定指令，接收被选定的视频控件相对应的第二视频流，并以指定布局将该第二视频流与本地采集的第一视频流合成为第三视频流；a synthesis module: responsive to a selected instruction acting on at least one of the video controls, receiving a second video stream corresponding to the selected video control, and the second video stream and the locally acquired A video stream is synthesized into a third video stream;
上传模块：用于将所述第三视频流上传到服务器，以使得所述服务器向直播间的每个用户推送所述第三视频流。The uploading module is configured to upload the third video stream to the server, so that the server pushes the third video stream to each user in the live broadcast room.
相应的，本发明还提供一种执行、实现上述直播间视频流合成控制方法的终端设备，包括处理器、存储器，所述处理器用于调用执行以程序形式存储于所述存储器的上述直播间视频流合成控制方法任意一项所述方法的步骤。Correspondingly, the present invention further provides a terminal device for performing and implementing the above-mentioned live video stream synthesis control method, comprising a processor and a memory, wherein the processor is configured to invoke to execute the live broadcast video stored in the memory in a program form. Flow synthesis control method The steps of any of the methods described.
与现有技术相比，本发明具备如下优点：Compared with the prior art, the present invention has the following advantages:
(1)本发明将主播终端配置为服务器终端，用以接收第二终端采集后直接发送至主播终端的第二视频流，再由主播终端将所有视频流合成推送至直播间的视频流，直播间的观众用户看到的视频流与主播终端上的视频流一致，保证了视频流的同时性；(1) The present invention configures the anchor terminal as a server terminal for receiving the second video stream directly sent by the second terminal to the anchor terminal, and then the allcast video stream is synthesized and pushed by the anchor terminal to the video stream of the live broadcast room, and the live broadcast is performed. The video stream seen by the viewer user is consistent with the video stream on the anchor terminal, ensuring the simultaneity of the video stream;
(2)同时，利用长连接保持主播终端与第二终端的通信链路，对于多个来源的第二视频流，在主播终端上选定连接任意一个第二视频流后，接收被选定的第二视频流，并且选择连接任意一个第二视频流的过程可通过随机方式选择且展示在直播画面中，增强主播与观众用户的互动效果；(2) At the same time, the long link is used to maintain the communication link between the anchor terminal and the second terminal. For the second video stream of multiple sources, after selecting any one of the second video streams on the anchor terminal, the reception is selected. The second video stream, and the process of selecting to connect any one of the second video streams can be selected in a random manner and displayed in the live broadcast screen to enhance the interaction effect between the anchor and the viewer user;
(3)再者，主播可调整合成的视频流的画面排版布局及音频效果，以更好地突出直播内容，营造更好的直播效果。(3) Furthermore, the anchor can adjust the screen layout and audio effects of the synthesized video stream to better highlight the live content and create a better live broadcast effect.
显然，上述有关本发明优点的描述是概括性的，更多的优点描述将体现在后续的实施例揭示中，以及，本领域技术人员也可以本发明所揭示的内容合理地发现本发明的其他诸多优点。It is apparent that the above description of the advantages of the present invention is general, and more advantages will be described in the following embodiments, and those skilled in the art can also reasonably discover other aspects of the present invention. Many advantages.
本发明附加的方面和优点将在下面的描述中部分给出，这些将从下面的描述中变得明显，或通过本发明的实践了解到。The additional aspects and advantages of the invention will be set forth in part in the description which follows.
【附图说明】[Description of the Drawings]
图1为本发明直播间视频流合成控制方法的一实施例流程示意图；1 is a schematic flow chart of an embodiment of a method for controlling video stream synthesis between live broadcasts according to the present invention;
图2为本发明主播终端开启多源直播模式后的直播界面示意图；2 is a schematic diagram of a live broadcast interface after the multi-source live broadcast mode is enabled on the anchor terminal of the present invention;
图3为本发明选定连接2个第二视频流后的第三视频流的排版布局示意 图；3 is a schematic layout diagram of a third video stream after selecting two second video streams according to the present invention Figure
图4为本发明直播间视频流合成控制装置的一实施例示意图；4 is a schematic diagram of an embodiment of a video stream synthesis control device between broadcasts according to the present invention;
图5是实现本发明所述直播间视频流合成控制方法的系统示意图。FIG. 5 is a schematic diagram of a system for implementing a video stream synthesis control method between broadcasts according to the present invention.
【具体实施方式】【detailed description】
下面结合附图和示例性实施例对本发明作进一步地描述，其中附图中相同的标号全部指的是相同的部件。此外，如果已知技术的详细描述对于示出本发明的特征是不必要的，则将其省略。The invention is further described in the following with reference to the drawings and exemplary embodiments, wherein like reference numerals refer to the same parts throughout. Further, if a detailed description of a known technique is not necessary to show the features of the present invention, it will be omitted.
本技术领域技术人员可以理解，除非特意声明，这里使用的单数形式“一”、“一个”、“所述”和“该”也可包括复数形式。应该进一步理解的是，本发明的说明书中使用的措辞“包括”是指存在所述特征、整数、步骤、操作、元件和/或组件，但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元件、组件和/或它们的组。应该理解，当我们称元件被“连接”或“耦接”到另一元件时，它可以直接连接或耦接到其他元件，或者也可以存在中间元件。此外，这里使用的“连接”或“耦接”可以包括无线连接或无线耦接。这里使用的措辞“和/或”包括一个或更多个相关联的列出项的全部或任一单元和全部组合。The singular forms "a", "an", "the" It is to be understood that the phrase "comprise" or "an" Integers, steps, operations, components, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element. Further, "connected" or "coupled" as used herein may include either a wireless connection or a wireless coupling. The phrase "and/or" used herein includes all or any one and all combinations of one or more of the associated listed.
本技术领域技术人员可以理解，除非另外定义，这里使用的所有术语(包括技术术语和科学术语)，具有与本发明所属领域中的普通技术人员的一般理解相同的意义。还应该理解的是，诸如通用字典中定义的那些术语，应该被理解为具有与现有技术的上下文中的意义一致的意义，并且除非像这里一样被特定定义，否则不会用理想化或过于正式的含义来解释。Those skilled in the art will appreciate that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention belongs, unless otherwise defined. It should also be understood that terms such as those defined in a general dictionary should be understood to have meaning consistent with the meaning in the context of the prior art, and will not be idealized or excessive unless specifically defined as here. The formal meaning is explained.
本技术领域技术人员可以理解，这里所使用的“终端”、“终端设备”既包括无线信号接收器的设备，其仅具备无发射能力的无线信号接收器的设备，又包括接收和发射硬件的设备，其具有能够在双向通信链路上，执行双向通信的接收和发射硬件的设备。这种设备可以包括：蜂窝或其他通信设备，其具有单线路显示器或多线路显示器或没有多线路显示器的蜂窝或其他通信设备；PCS(Personal Communications Service，个人通信系统)，其可以组合语音、数据处理、传真和/或数据通信能力；PDA(Personal Digital Assistant，个人数字助理)，其可以包括射频接收器、寻呼机、互联网/内联网访问、网 络浏览器、记事本、日历和/或GPS(Global Positioning System，全球定位系统)接收器；常规膝上型和/或掌上型计算机或其他设备，其具有和/或包括射频接收器的常规膝上型和/或掌上型计算机或其他设备。这里所使用的“终端”、“终端设备”可以是便携式、可运输、安装在交通工具(航空、海运和/或陆地)中的，或者适合于和/或配置为在本地运行，和/或以分布形式，运行在地球和/或空间的任何其他位置运行。这里所使用的“终端”、“终端设备”还可以是通信终端、上网终端、音乐/视频播放终端，例如可以是PDA、MID(Mobile Internet Device，移动互联网设备)和/或具有音乐/视频播放功能的移动电话，也可以是智能电视、机顶盒等设备。Those skilled in the art can understand that the "terminal" and "terminal device" used herein include both a wireless signal receiver device, a device having only a wireless signal receiver without a transmitting capability, and a receiving and transmitting hardware. A device having a device capable of performing two-way communication receiving and transmitting hardware on a two-way communication link. Such devices may include cellular or other communication devices having a single line display or a multi-line display or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service), which may combine voice, data processing, facsimile and / or data communications capabilities; PDA (personal digital Assistant, personal digital assistants), which may include network access to a radio frequency receiver, a pager, Internet / web browser, notepad, calendar and / or GPS (Global Positioning System (Global Positioning System) receiver; conventional laptop and/or palmtop computer or other device having a conventional laptop and/or palmtop computer or other device that includes and/or includes a radio frequency receiver. As used herein, "terminal", "terminal device" may be portable, transportable, installed in a vehicle (aviation, sea and/or land), or adapted and/or configured to operate locally, and/or Run in any other location on the Earth and/or space in a distributed form. The "terminal" and "terminal device" used herein may also be a communication terminal, an internet terminal, a music/video playing terminal, and may be, for example, a PDA, a MID (Mobile Internet Device), and/or have a music/video playback. Functional mobile phones can also be smart TVs, set-top boxes and other devices.
本技术领域技术人员可以理解，这里所使用的远端网络设备，其包括但不限于计算机、网络主机、单个网络服务器、多个网络服务器集或多个服务器构成的云。在此，云由基于云计算(Cloud Computing)的大量计算机或网络服务器构成，其中，云计算是分布式计算的一种，由一群松散耦合的计算机集组成的一个超级虚拟计算机。本发明的实施例中，远端网络设备、终端设备与WNS服务器之间可通过任何通信方式实现通信，包括但不限于，基于3GPP、LTE、WIMAX的移动通信、基于TCP/IP、UDP协议的计算机网络通信以及基于蓝牙、红外传输标准的近距无线传输方式。Those skilled in the art can understand that the remote network device used herein includes, but is not limited to, a computer, a network host, a single network server, a plurality of network server sets, or a cloud composed of multiple servers. Here, the cloud is composed of a large number of computers or network servers based on Cloud Computing, which is a kind of distributed computing, a super virtual computer composed of a group of loosely coupled computers. In the embodiment of the present invention, the communication between the remote network device, the terminal device and the WNS server can be implemented by any communication method, including but not limited to, mobile communication based on 3GPP, LTE, WIMAX, TCP/IP, UDP protocol. Computer network communication and short-range wireless transmission based on Bluetooth and infrared transmission standards.
本技术领域技术人员可以理解，本发明所述的用户界面/操作界面泛指能够用于向智能终端发送控制指令的显示界面，例如，可以为Andro i d系统的设置页面中的一个选项(或按键，由所述应用程序添加其中，下同)，也可以是从桌面呼出的通知栏或者交互页面中的一个选项，还可以是所述应用程序的一个活动组件所构造的页面中的一个选项。虽然上面已经示出了本发明的一些示例性实施例，但是本领域的技术人员将理解，在不脱离本发明的原理或精神的情况下，可以对这些示例性实施例做出改变，本发明的范围由权利要求及其等同物限定。Those skilled in the art can understand that the user interface/operation interface of the present invention generally refers to a display interface that can be used to send control instructions to the smart terminal, for example, an option in the setting page of the Andro i d system (or a button, added by the application, the same below), may also be an option in the notification bar or interactive page that is called from the desktop, or may be an option in a page constructed by an active component of the application. . While some exemplary embodiments of the invention have been shown in the foregoing, the embodiments of the invention may The scope is defined by the claims and their equivalents.
本领域技术人员应当理解，本发明所称的“应用”、“应用程序”、“应用软件”以及类似表述的概念，是业内技术人员所公知的相同概念，是指由一系列计算机指令及相关数据资源有机构造的适于电子运行的计算机软件。除非特别指定，这种命名本身不受编程语言种类、级别，也不受其赖以运行的操作系统或平台所限制。理所当然地，此类概念也不受任何形式的终端所 限制。Those skilled in the art should understand that the concepts of "application", "application", "application software" and similar expressions as used in the present invention are the same concepts well known to those skilled in the art, and are referred to by a series of computer instructions and related. Data software is an organically constructed computer software suitable for electronic operation. Unless otherwise specified, the naming itself is not limited by the type and level of programming language, nor by the operating system or platform on which it operates. Of course, such concepts are also not subject to any form of terminal limit.
直播间：本发明所述直播间包括以下含义，①一种基于网络直播平台创建的一个虚拟空间(或虚拟房间)，直播间一般由主播客户端创建并连接有多个观看客户端，即直播间中包括了主播及多个观众，位于该虚拟空间内的观看客户端可以观看主播客户端的直播内容，同时主播客户端的用户与观看客户端的用户、观看客户端的用户与观看客户端的用户之间还可以进行语音、图片、文字或赠送电子赠品的互动；②一种以群组方式将用户聚合在一起的即时通讯平台，例如视频会议系统，用户通过登录客户端的方式进入群组，用户在群组内以群组成员的身份存在，同一个群组内包含有多个群组成员，用户可任意加入或退出群组，在群组内，可以进行文字、语音、视频等多种互动。The live broadcast room includes the following meanings: 1 a virtual space (or virtual room) created based on the webcast platform. The live broadcast room is generally created by the anchor client and connected to multiple viewing clients, that is, live broadcast. The anchor and the plurality of viewers are included in the room, and the viewing client located in the virtual space can watch the live content of the anchor client, and the user of the anchor client and the user who views the client, the user who views the client, and the user who views the client. You can interact with voice, pictures, text or electronic gifts; 2 an instant messaging platform that aggregates users together in groups, such as a video conferencing system, where users log in to the client to enter the group, and the user is in the group. The group exists as a member of the group. The same group contains multiple group members. The user can join or leave the group arbitrarily. Within the group, various interactions such as text, voice, and video can be performed.
如图1所示是本发明直播间视频流合成控制方法的一实施例流程示意图，包括如下步骤：FIG. 1 is a schematic flowchart of an embodiment of a method for controlling video stream synthesis in a live broadcast according to the present invention, including the following steps:
本实施例中，为了便于说明，所述第一视频流为以主播身份开启直播的主播用户终端所采集的视频流，所述第二视频流为与主播终端连接的第二用户终端所采集的视频流。In this embodiment, for convenience of description, the first video stream is a video stream collected by an anchor user terminal that starts a live broadcast by an anchor identity, and the second video stream is collected by a second user terminal connected to the anchor terminal. Video stream.
步骤S100：显示与第二视频流个数相对应的一个或多个视频控件。Step S100: Display one or more video controls corresponding to the number of second video streams.
多源直播是指在一个直播画面中同时存在有多个来源的视频流的直播方式，该多个来源的视频流可以是主播用户利用多个拍摄设备采集的多个视频流或主播用户邀请其他用户一起直播时的多个用户终端采集的视频流；在本实施例中，主播用户开启多源直播模式，第二终端申请与主播终端连接以进行多源直播，此时的主播终端被配置为服务器终端，用于接收第二终端发送而来的第二视频流，在主播用户确认接收第二视频流前，第二终端与主播终端以长连接的形式保持双方的通信链路连接，在长连接保持期间，如果没有视频流数据的传输，需要第二终端每隔一段时间给主播终端发送一个心跳数据包以保持该长链接，在主播用户确认接收第二视频流后第二终端通过所述通信链路将第二视频流发送至主播终端。Multi-source live broadcast refers to a live broadcast mode in which a plurality of video streams of multiple sources exist simultaneously in one live broadcast screen, and the video streams of the multiple sources may be multiple video streams collected by the anchor user using multiple photographing devices or the host user invites other The video stream collected by the user terminals when the user is live together; in this embodiment, the anchor user starts the multi-source live broadcast mode, and the second terminal requests to connect with the anchor terminal to perform multi-source live broadcast, and the anchor terminal at this time is configured as The server terminal is configured to receive the second video stream sent by the second terminal, and before the anchor user confirms to receive the second video stream, the second terminal and the anchor terminal maintain a communication link connection between the two parties in a long connection manner. During the connection hold period, if there is no transmission of the video stream data, the second terminal needs to send a heartbeat data packet to the anchor terminal at intervals to maintain the long link, and after the anchor user confirms receiving the second video stream, the second terminal passes the The communication link sends the second video stream to the anchor terminal.
本实施例中，与主播终端保持长连接的一个或多个第二终端(也即第二视频流)以视频控件的形式显示在主播终端的用户界面上，每一个视频控件对应一个第二视频流的选择入口，并且在所述视频控件中还包含用于区分多 个第二视频流的特征信息，通过选择视频控件可以触发接收对应的第二视频流，优选的，所述特征信息为第二终端用户在直播平台上的用户名。In this embodiment, one or more second terminals (ie, the second video stream) that maintain a long connection with the anchor terminal are displayed on the user interface of the anchor terminal in the form of a video control, and each video control corresponds to a second video. a selection entry for the stream, and also included in the video control for distinguishing The feature information of the second video stream can be triggered to receive the corresponding second video stream by selecting a video control. Preferably, the feature information is a user name of the second terminal user on the live broadcast platform.
如图2所示是主播终端开启多源直播模式后的直播界面示意图，图2中示出4个第二终端申请与主播终端连接以进行多源直播，在直播界面的上方以4个视频控件显示与主播终端保持长连接的4个第二终端的第二视频流，并且在视频控件中添加了用于区分不同第二视频流的特征信息，即第二终端用户在直播平台上的用户名，如图2中的“虎牙第一妖男”、“老邢姑娘sds”、“小人鱼老司机”及“每天不一样…”。FIG. 2 is a schematic diagram of a live broadcast interface after the multi-source live broadcast mode is enabled on the anchor terminal. FIG. 2 shows that four second terminals apply for connection with the anchor terminal for multi-source live broadcast, and four video controls are arranged above the live broadcast interface. Displaying a second video stream of the 4 second terminals that are connected to the anchor terminal, and adding feature information for distinguishing different second video streams, that is, the user name of the second terminal user on the live broadcast platform As shown in Figure 2, "The first demon male of Huya", "Old Xing girl sds", "The old civet old driver" and "Different every day...".
步骤S200：响应作用于所述视频控件的至少一个的选定指令，接收被选定的视频控件相对应的第二视频流，并以指定布局将该第二视频流与本机采集的第一视频流合成为第三视频流。Step S200: Receive a second video stream corresponding to the selected video control in response to the selected instruction acting on at least one of the video controls, and first capture the second video stream with the local device in a specified layout. The video stream is synthesized into a third video stream.
在显示于主播终端用户界面上的视频控件，当发生作用于所述视频控件的至少一个的选定操作时，触发相应的选定指令，主播终端响应于所述选定指令开始接收被选定的视频控件相对应的第二视频流，并以指定布局将该第二视频流与本机采集的第一视频流合成为第三视频流。The video control displayed on the anchor terminal user interface triggers a corresponding selected command when a selected operation occurs on at least one of the video controls, and the anchor terminal starts receiving in response to the selected command The video control corresponds to the second video stream, and combines the second video stream with the locally collected first video stream into a third video stream in a specified layout.
所述选定指令的一种实施方式是，所述选定指令为由主播用户选择至少一个视频控件的选定指令，该方式下主播用户持有选择任何一个或多个第二终端用户所对应的视频控件的控制权，主播用户可以根据自己对第二终端用户的喜好选择一个或多个第二终端用户所对应的视频控件，以接收该视频控件对应的第二视频流；所述选定指令的另一种实施方式，为了增强主播与用户的活动，活跃直播间气氛，并且让更多用户参与其中，所述选定指定为根据预设规则选择至少一个视频控件的选定指令，优选的，所述预设规则为从连接的多个第二视频流对应的多个视频控件中以随机方式选定一个或多个视频控件，以接收该视频控件对应的第二视频流，随机选择的过程同时也表现在第三视频流中，在保持长连接的第二视频流的数量有一定限制时，该方式可以增强主播跟用户之间的互动效果。An embodiment of the selected instruction is that the selected instruction is a selected instruction for selecting at least one video control by an anchor user, wherein the anchor user holds the selection of any one or more second terminal users. Controlling the video control, the anchor user may select one or more video controls corresponding to the second terminal user according to the preference of the second terminal user to receive the second video stream corresponding to the video control; Another implementation of the instruction, in order to enhance the activity of the anchor and the user, activate the atmosphere between the live broadcasts, and let more users participate, the selection is designated as selecting a selected instruction of the at least one video control according to a preset rule, preferably The preset rule is that one or more video controls are randomly selected from a plurality of video controls corresponding to the plurality of connected second video streams to receive the second video stream corresponding to the video control, and randomly selected The process also appears in the third video stream, which can enhance the master when there is a limit on the number of second video streams that maintain a long connection. With interaction effects between users.
进一步的，所述选定指令还包括由主播用户选用预设的多个排版布局之一作为第三视频流的指定布局的选定指令。在选定了连接的第二视频流所对应的视频控件后，需由主播用户选择第三视频流的排版布局，其中，主播终端预设了多个第三视频流的排版布局，包括连接1个、2个、3个第二视频流 时的排版布局，并且所述排版布局包括第一、第二视频流的图像流在合成的第三视频流中的图像位置、图像大小、图像倾斜角度的排版布局，如图3所示是选定连接2个第二视频流(“虎牙第一妖男”和“老邢姑娘sds”)后的第三视频流的排版布局示意图。Further, the selected instruction further includes a selected instruction selected by the anchor user as one of a plurality of preset layout layouts as the specified layout of the third video stream. After the video control corresponding to the connected second video stream is selected, the anchor user needs to select the layout layout of the third video stream, wherein the anchor terminal presets the layout layout of the plurality of third video streams, including the connection 1 , 2, 3 second video streams a layout layout of the time, and the layout layout includes an image position of the image streams of the first and second video streams in the synthesized third video stream, an image size, and a layout layout of the image tilt angle, as shown in FIG. A layout layout of the third video stream after connecting two second video streams ("the first demon male" and "the old sds").
进一步的，在根据所选定的排版布局作为第三视频流的指定布局显示合成的第三视频流后，主播用户还可以根据需要修改第三视频流的排版布局，主播终端响应作用于所述第三视频流在用户界面上的排版布局的编辑指令，改变第三视频流的图像排版布局效果，其中，针对第三视频流的排版布局的编辑包括针对第一、第二视频流的图像流在合成的第三视频流中的图像位置、图像大小、图像倾斜角度的排版布局，如图3所示，主播用户选择其中一个位于直播画面右上角的第二视频流，该第二视频流的画面以框线突出显示已被选定状态，主播用户可以通过拖动框线以修改该第二视频流的图形流的图像大小、通过点击该第二视频流画面下方从左往右数起的第二个按钮以修改该第二视频流的图形流的图像倾斜角度。Further, after displaying the synthesized third video stream according to the selected layout layout as the specified layout of the third video stream, the anchor user may further modify the layout layout of the third video stream, and the anchor terminal response acts on the Editing instructions of the layout of the third video stream on the user interface, changing the image layout layout effect of the third video stream, wherein the editing of the layout layout for the third video stream includes image streams for the first and second video streams The layout position of the image position, the image size, and the tilt angle of the image in the synthesized third video stream, as shown in FIG. 3, the anchor user selects one of the second video streams located in the upper right corner of the live video screen, and the second video stream The screen is highlighted by a frame line, and the anchor user can modify the image size of the graphics stream of the second video stream by dragging the frame line, and counting from left to right by clicking below the second video stream screen. The second button is to modify the image tilt angle of the graphics stream of the second video stream.
另外，本实施例还包括步骤：响应作用于第二视频流的音频流的操作指令，打开/关闭第二视频流的音频流或调整第二视频流的音频流的大小。主播用户还可以根据需要调整第二视频流的音频流，其中包括打开/关闭第二视频流的音频流和调整第二视频流的音频流的大小，一般地，在第二视频流的图像画面中设有该第二视频流的音频流的操作按钮，如图3所示的直播画面右上角的第二视频流的画面下方从左往右数起的第三个按钮，主播用户可以操作该按钮以打开/关闭第二视频流的音频流和调整第二视频流的音频流的大小。In addition, the embodiment further includes the step of: turning on/off the audio stream of the second video stream or adjusting the size of the audio stream of the second video stream in response to an operation instruction of the audio stream acting on the second video stream. The anchor user may also adjust the audio stream of the second video stream as needed, including turning on/off the audio stream of the second video stream and adjusting the size of the audio stream of the second video stream, generally, the image frame of the second video stream The operation button of the audio stream of the second video stream is provided, and the third button from the left to the right of the screen of the second video stream in the upper right corner of the live video screen shown in FIG. 3 is used, and the anchor user can operate the button. Button to turn on/off the audio stream of the second video stream and adjust the size of the audio stream of the second video stream.
步骤S300：将所述第三视频流上传到服务器，以使得所述服务器向直播间的每个用户推送所述第三视频流。Step S300: Upload the third video stream to a server, so that the server pushes the third video stream to each user in the live broadcast room.
在合成第三视频流后，主播终端将第三视频流上传到直播平台的服务器，以使得所述服务器向直播间的每个用户推送所述第三视频流，在本实施例中，所述直播终端接收第二视频流的过程与采集本地的第一视频流的过程与合成第三视频流的过程与上传第三视频流的过程并行工作，并行工作是指同一时刻或同一时间间隔内完成两种或两种以上性质相同或不相同的工作，并行工作具有①时间重叠性：相邻处理过程在时间上错开，轮流重叠使用同一套硬 件的各部分；②资源共享性：让多个用户按照一定的时间顺序轮流使用同一套资源，提高资源利用率；③资源重复：重复设置硬件资源，提高硬件可靠性和性能。具体的，对于本发明实施例而言，并行工作是指主播终端在接收第二直频流时，可以同步/同步采集第一视频流，并同时/同步将所采集的第一视频流和所接收到的第二视频流合并成第三视频流，在此期间，还包括对第三视频流的排版布局的修改，对第二视频流的音频流的操作，且同时/同步将第三视频流上传到服务器，保证了直播视频流的同时/同步性。After synthesizing the third video stream, the anchor terminal uploads the third video stream to the server of the live broadcast platform, so that the server pushes the third video stream to each user in the live broadcast room. In this embodiment, the The process of receiving the second video stream by the live broadcast terminal and the process of collecting the local first video stream and the process of synthesizing the third video stream work in parallel with the process of uploading the third video stream, and the parallel work refers to completion at the same time or at the same time interval. Two or more jobs of the same or different nature, parallel work has 1 time overlap: adjacent processes are staggered in time, alternately using the same set of hard Each part of the piece; 2 resource sharing: let multiple users use the same set of resources in turn in a certain time sequence to improve resource utilization; 3 resource duplication: repeatedly set hardware resources to improve hardware reliability and performance. Specifically, for the embodiment of the present invention, the parallel operation means that the anchor terminal can synchronously/synchronize the first video stream when receiving the second direct stream, and simultaneously/synchronize the collected first video stream and the The received second video stream is merged into a third video stream, during which time, the modification of the layout layout of the third video stream, the operation of the audio stream of the second video stream, and the simultaneous/synchronization of the third video are also included. The stream is uploaded to the server to ensure the simultaneous/synchronization of the live video stream.
相应的，如图2所示是本发明直播间视频流合成控制装置的一实施例示意图，包括：Correspondingly, as shown in FIG. 2 is a schematic diagram of an embodiment of a live video stream synthesis control apparatus according to the present invention, including:
调用模块100：用于显示与第二视频流个数相对应的一个或多个视频控件。The calling module 100 is configured to display one or more video controls corresponding to the number of the second video streams.
在本实施例中，主播用户开启多源直播模式，第二终端申请与主播终端连接以进行多源直播，此时的主播终端被配置为服务器终端，用于接收第二终端发送而来的第二视频流，在主播用户确认接收第二视频流前，第二终端与主播终端以长连接的形式保持双方的通信链路连接，在长连接保持期间，如果没有视频流数据的传输，需要第二终端每隔一段时间给主播终端发送一个心跳数据包以保持该长链接，在主播用户确认接收第二视频流后第二终端通过所述通信链路将第二视频流发送至主播终端，此时调用模块100将与主播终端保持长连接的一个或多个第二终端(也即第二视频流)以视频控件的形式显示在主播终端的用户界面上，并且，在所述视频控件中还包含用于区分多个第二视频流的特征信息，优选的，所述特征信息为第二终端用户在直播平台上的用户名。In this embodiment, the anchor user starts the multi-source live broadcast mode, and the second terminal applies for connection with the anchor terminal to perform multi-source live broadcast. The anchor terminal at this time is configured as a server terminal, and is configured to receive the second terminal. In the second video stream, before the anchor user confirms to receive the second video stream, the second terminal and the anchor terminal maintain a communication link connection between the two parties in a long connection manner, and during the long connection hold period, if there is no video stream data transmission, the first The second terminal sends a heartbeat data packet to the anchor terminal at intervals to maintain the long link. After the anchor user confirms receiving the second video stream, the second terminal sends the second video stream to the anchor terminal through the communication link. The calling module 100 displays one or more second terminals (ie, the second video stream) that are connected to the anchor terminal in a long form in the form of a video control on the user interface of the anchor terminal, and also in the video control. The feature information is used to distinguish the plurality of second video streams. Preferably, the feature information is a user of the second terminal user on the live broadcast platform. .
合成模块200：用于响应作用于所述视频控件的至少一个的选定指令，接收被选定的视频控件相对应的第二视频流，并以指定布局将该第二视频流与本地采集的第一视频流合成为第三视频流。The synthesizing module 200 is configured to receive a second video stream corresponding to the selected video control in response to the selected instruction acting on the at least one of the video controls, and to locally capture the second video stream with a specified layout The first video stream is synthesized into a third video stream.
在调用模块100将视频控件显示于主播终端用户界面上后，当发生作用于所述视频控件的至少一个的选定操作时，触发相应的选定指令，合成模块200响应于所述选定指令开始接收被选定的视频控件相对应的第二视频流，并以指定布局将该第二视频流与本机采集的第一视频流合成为第三视频流。After the calling module 100 displays the video control on the anchor terminal user interface, when a selected operation occurs on at least one of the video controls, a corresponding selected instruction is triggered, and the synthesizing module 200 is responsive to the selected instruction. Start receiving a second video stream corresponding to the selected video control, and synthesizing the second video stream with the locally collected first video stream into a third video stream in a specified layout.
上传模块300：用于将所述第三视频流上传到服务器，以使得所述服务 器向直播间的每个用户推送所述第三视频流。Uploading module 300: for uploading the third video stream to a server, so that the service The device pushes the third video stream to each user in the live room.
在合成模块200合成第三视频流的同时，上传模块300将第三视频流上传到直播平台的服务器，以使得所述服务器向直播间的每个用户推送所述第三视频流，在本实施例中，所述合成模块200接收第二视频流的过程与采集本地的第一视频流的过程与合成第三视频流的过程与上传模块300上传第三视频流的过程并行工作。并行工作是指合成模块200在接收第二直频流时，可以同步/同步采集第一视频流，并同时/同步将所采集的第一视频流和所接收到的第二视频流合并成第三视频流，在此期间，还包括对第三视频流的排版布局的修改，对第二视频流的音频流的操作，且上传模块300同时/同步将第三视频流上传到服务器，保证了直播视频流的同时/同步性。While the synthesizing module 200 synthesizes the third video stream, the uploading module 300 uploads the third video stream to the server of the live broadcast platform, so that the server pushes the third video stream to each user in the live broadcast room, in the implementation. In an example, the process of receiving the second video stream by the synthesizing module 200 and the process of collecting the local first video stream and the process of synthesizing the third video stream work in parallel with the process of uploading the third video stream by the uploading module 300. The parallel operation means that the synthesizing module 200 can synchronously/synchronize the first video stream when receiving the second direct stream, and simultaneously/synchronize the collected first video stream and the received second video stream into a first The three video streams, during which the modification of the layout of the third video stream, the operation of the audio stream of the second video stream, and the uploading module 300 simultaneously/synchronizing the third video stream to the server are ensured. Simultaneous/synchronization of live video streams.
另外，本发明还提供一种终端设备，包括存储器、处理器，所述存储器用于存储上述方法实现过程中产生的候选中间数据以及结果数据，所述处理器用于调用执行以程序形式存储于所述存储器的所述直播间视频流合成控制方法的任意一项步骤。In addition, the present invention further provides a terminal device, including a memory, a processor, configured to store candidate intermediate data and result data generated in the implementation of the foregoing method, where the processor is used to execute execution in a program form. Any one of the steps of the live stream video stream synthesis control method of the memory.
如图5所示是实现本发明所述直播间视频流合成控制方法的系统示意图，主播用户与第二终端用户通过身份验证后保持主播终端与第二终端的长连接，然后第二终端将第二视频流发送至主播终端，主播终端再同时将本地采集的视频流和接收到的第二视频流合成第三视频流后上传到直播平台的视频流服务器，再通过直播平台的视频流服务器将第三视频流推送给直播间的各个用户。FIG. 5 is a schematic diagram of a system for implementing a video stream synthesis control method according to the present invention. After the anchor user and the second terminal user pass the identity verification, the long connection between the anchor terminal and the second terminal is maintained, and then the second terminal will be connected. The second video stream is sent to the anchor terminal, and the anchor terminal simultaneously synthesizes the locally collected video stream and the received second video stream into a third video stream, and then uploads the video stream to the video stream server of the live broadcast platform, and then passes the video stream server of the live broadcast platform. The third video stream is pushed to each user in the live broadcast room.
本发明将主播终端配置为服务器终端，用以接收第二终端采集后直接发送至主播终端的第二视频流，再由主播终端将所有视频流合成推送至直播间的视频流，直播间的观众用户看到的视频流与主播终端上的视频流一致，保证了视频流的同时性；同时，利用长连接保持主播终端与第二终端的通信链路，对于多个来源的第二视频流，在主播终端上选定连接任意一个第二视频流后，接收被选定的第二视频流，并且选择连接任意一个第二视频流的过程可通过随机方式选择且展示在直播画面中，增强主播与观众用户的互动效果；再者，主播可调整合成的视频流的画面排版布局及音频效果，以更好地突出直播内容，营造更好的直播效果。The present invention configures the anchor terminal as a server terminal for receiving the second video stream directly sent by the second terminal to the anchor terminal, and then the allcast video stream is synthesized and pushed by the anchor terminal to the video stream in the live broadcast room, and the audience in the live broadcast room The video stream seen by the user is consistent with the video stream on the anchor terminal, ensuring the simultaneity of the video stream; at the same time, the long link is used to maintain the communication link between the anchor terminal and the second terminal, and for the second video stream of multiple sources, After the connection of any one of the second video streams is selected on the anchor terminal, the process of receiving the selected second video stream and selecting to connect any one of the second video streams may be selected in a random manner and displayed on the live broadcast screen to enhance the anchor. The interactive effect with the viewer user; in addition, the anchor can adjust the screen layout and audio effects of the synthesized video stream to better highlight the live content and create a better live broadcast effect.
虽然上面已经示出了本发明的一些示例性实施例，但是本领域的技术人 员将理解，在不脱离本发明的原理或精神的情况下，可以对这些示例性实施例做出改变，本发明的范围由权利要求及其等同物限定。 Although some exemplary embodiments of the present invention have been shown above, those skilled in the art It is to be understood that modifications may be made to the exemplary embodiments without departing from the spirit and scope of the invention.

Claims (10)

  1. 一种直播间视频流合成控制方法，其特征在于，包括如下步骤：A method for controlling video stream synthesis in a live broadcast, comprising the steps of:
    显示与第二视频流个数相对应的一个或多个视频控件；Displaying one or more video controls corresponding to the number of second video streams;
    响应作用于所述视频控件的至少一个的选定指令，接收被选定的视频控件相对应的第二视频流，并以指定布局将该第二视频流与本地采集的第一视频流合成为第三视频流；Responding to a selected instruction acting on at least one of the video controls, receiving a second video stream corresponding to the selected video control, and synthesizing the second video stream with the locally acquired first video stream in a specified layout as Third video stream;
    将所述第三视频流上传到服务器，以使得所述服务器向直播间的每个用户推送所述第三视频流。Uploading the third video stream to a server, so that the server pushes the third video stream to each user in the live room.
  2. 根据权利要求1所述的方法，其特征在于，所述第一、第二、第三视频流均包括图像流和音频流，所述第三视频流包括第一视频流、第二视频流至少之一的图像流，还包括该两者至少之一的音频流。The method according to claim 1, wherein the first, second, and third video streams each include an image stream and an audio stream, and the third video stream includes a first video stream and a second video stream. One of the image streams further includes an audio stream of at least one of the two.
  3. 根据权利要求1所述的方法，其特征在于，所述选定指令包括由主播用户选择至少一个视频控件的选定指令，和根据预设规则选择至少一个视频控件的选定指令。The method of claim 1 wherein said selected instructions comprise selected instructions for selecting at least one video control by an anchor user, and selecting selected instructions for at least one video control in accordance with a preset rule.
  4. 根据权利要求3所述的方法，其特征在于，所述选定指令还包括由主播用户选用预设的多个排版布局之一作为第三视频流的指定布局的选定指令。The method of claim 3 wherein said selected instructions further comprise selected ones of said plurality of layout layouts selected by the anchor user as the specified layout of the third video stream.
  5. 根据权利要求4所述的方法，其特征在于，所述排版布局包括第一、第二视频流的图像流在合成的第三视频流中的图像位置、图像大小、图像倾斜角度的排版布局。The method according to claim 4, wherein the layout layout comprises an image position of the image streams of the first and second video streams in the synthesized third video stream, an image size, and a layout layout of the image tilt angle.
  6. 根据权利要求5所述的方法，其特征在于，还包括步骤：The method of claim 5 further comprising the step of:
    响应作用于所述第三视频流在用户界面上的排版布局的编辑指令，改变第三视频流的图像排版布局效果。And responding to an editing instruction of the layout layout of the third video stream on the user interface, changing an image layout effect of the third video stream.
  7. 根据权利要求2所述的方法，其特征在于，还包括步骤：The method of claim 2 further comprising the step of:
    响应作用于第二视频流的音频流的操作指令，打开/关闭第二视频流的音频流或调整第二视频流的音频流的大小。The audio stream of the second video stream is turned on/off or the size of the audio stream of the second video stream is adjusted in response to an operation instruction of the audio stream acting on the second video stream.
  8. 根据权利要求1所述的方法，其特征在于，所述接收第二视频流的过程与采集本地的第一视频流的过程与合成第三视频流的过程与上传第三视频流的过程并行工作。 The method according to claim 1, wherein the process of receiving the second video stream and the process of collecting the local first video stream and the process of synthesizing the third video stream work in parallel with the process of uploading the third video stream .
  9. 一种直播间视频流合成控制装置，其特征在于，包括：A live stream video stream synthesis control device, comprising:
    调用模块：用于显示与第二视频流个数相对应的一个或多个视频控件；Calling module: for displaying one or more video controls corresponding to the number of second video streams;
    合成模块：用于响应作用于所述视频控件的至少一个的选定指令，接收被选定的视频控件相对应的第二视频流，并以指定布局将该第二视频流与本地采集的第一视频流合成为第三视频流；a synthesis module: responsive to a selected instruction acting on at least one of the video controls, receiving a second video stream corresponding to the selected video control, and the second video stream and the locally acquired A video stream is synthesized into a third video stream;
    上传模块：用于将所述第三视频流上传到服务器，以使得所述服务器向直播间的每个用户推送所述第三视频流。The uploading module is configured to upload the third video stream to the server, so that the server pushes the third video stream to each user in the live broadcast room.
  10. 一种终端设备，其特征在于，包括处理器、存储器，所述处理器用于调用执行以程序形式存储于所述存储器的如权利要求1～8任意一项所述方法的步骤。 A terminal device, comprising a processor, a memory, the processor for invoking a step of executing the method according to any one of claims 1 to 8 stored in the memory in a program form.
PCT/CN2017/107317 2016-11-22 2017-10-23 Control method, device, and terminal apparatus for synthesizing video stream of live streaming room WO2018095174A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201611049270.7A CN106658205B (en) 2016-11-22 2016-11-22 Live broadcast room video stream synthesis control method and device and terminal equipment
CN201611049270.7 2016-11-22

Publications (1)

Publication Number Publication Date
WO2018095174A1 true WO2018095174A1 (en) 2018-05-31

Family

ID=58812856

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/107317 WO2018095174A1 (en) 2016-11-22 2017-10-23 Control method, device, and terminal apparatus for synthesizing video stream of live streaming room

Country Status (2)

Country Link
CN (1) CN106658205B (en)
WO (1) WO2018095174A1 (en)

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110740346A (en) * 2019-10-23 2020-01-31 北京达佳互联信息技术有限公司 Video data processing method, device, server, terminal and storage medium
CN111182348A (en) * 2018-11-09 2020-05-19 阿里巴巴集团控股有限公司 Live broadcast picture display method and device
CN111814732A (en) * 2020-07-23 2020-10-23 上海优扬新媒信息技术有限公司 Identity verification method and device
CN111954054A (en) * 2020-06-05 2020-11-17 筑觉绘(上海)科技有限公司 Image processing method, system, storage medium and computer device
CN112188222A (en) * 2020-09-25 2021-01-05 吕庆新 Information acquisition processing device and application thereof, information transfer method, computer readable medium, and information acquisition processing method
CN112272283A (en) * 2020-09-14 2021-01-26 重庆国翔创新教学设备有限公司 Multi-party online learning method based on double cameras, client and server
CN112291502A (en) * 2020-02-24 2021-01-29 北京字节跳动网络技术有限公司 Information interaction method, device and system and electronic equipment
CN112312144A (en) * 2019-07-31 2021-02-02 腾讯科技（深圳）有限公司 Live broadcast method, device, equipment and storage medium
CN112954370A (en) * 2021-01-28 2021-06-11 深圳点猫科技有限公司 Encoding method, device and equipment for audio and video live broadcast
CN113163216A (en) * 2020-01-22 2021-07-23 华为技术有限公司 Live broadcast method and live broadcast system based on cloud mobile phone
CN113259692A (en) * 2020-02-11 2021-08-13 上海哔哩哔哩科技有限公司 Live broadcast interaction method and system
CN113453033A (en) * 2021-06-29 2021-09-28 广州方硅信息技术有限公司 Live broadcast room information transmission processing method and device, equipment and medium thereof
CN113556610A (en) * 2021-07-06 2021-10-26 广州方硅信息技术有限公司 Video synthesis control method, device, apparatus and medium
CN113727164A (en) * 2020-05-26 2021-11-30 百度在线网络技术（北京）有限公司 Live broadcast room entrance display method and device, electronic equipment and storage medium
CN114051170A (en) * 2021-11-16 2022-02-15 北京达佳互联信息技术有限公司 Live broadcast processing method and device, electronic equipment and computer readable storage medium
CN114466207A (en) * 2022-01-18 2022-05-10 阿里巴巴（中国）有限公司 Live broadcast control method and computer storage medium
CN114765695A (en) * 2021-01-15 2022-07-19 北京字节跳动网络技术有限公司 Live broadcast data processing method, device, equipment and medium
CN116708867A (en) * 2023-08-07 2023-09-05 深圳市亿晟科技有限公司 Live broadcast data processing method, device, equipment and storage medium
EP4220368A4 (en) * 2021-05-13 2024-03-13 Tencent Tech Shenzhen Co Ltd Multimedia data processing method and apparatus, and device, computer-readable storage medium and computer program product
CN113727164B (en) * 2020-05-26 2024-04-26 百度在线网络技术（北京）有限公司 Live broadcasting room entrance display method and device, electronic equipment and storage medium

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106658205B (en) * 2016-11-22 2020-09-04 广州华多网络科技有限公司 Live broadcast room video stream synthesis control method and device and terminal equipment
US20180336645A1 (en) * 2017-05-22 2018-11-22 Google Inc. Using machine learning to recommend live-stream content
CN107277636B (en) * 2017-06-15 2020-04-03 广州华多网络科技有限公司 Interaction method, user side, anchor side and system in live broadcast process
CN107333162B (en) * 2017-06-26 2020-08-11 广州华多网络科技有限公司 Method and device for playing live video
CN107360160A (en) * 2017-07-12 2017-11-17 广州华多网络科技有限公司 live video and animation fusion method, device and terminal device
CN107396166A (en) * 2017-08-07 2017-11-24 北京小米移动软件有限公司 The method and device of live middle display video
CN107566878B (en) * 2017-08-07 2020-09-08 北京小米移动软件有限公司 Method and device for displaying pictures in live broadcast
CN107659825B (en) * 2017-09-12 2019-01-29 武汉斗鱼网络科技有限公司 A kind of method, apparatus, server, main broadcaster end and medium that live video is retained
CN107948756B (en) * 2017-11-22 2020-07-24 广州华多网络科技有限公司 Video synthesis control method and device and corresponding terminal
CN109871115B (en) * 2017-12-04 2021-09-17 腾讯科技（深圳）有限公司 Control method, device and computer readable storage medium in multimedia interaction
CN108055577A (en) * 2017-12-18 2018-05-18 北京奇艺世纪科技有限公司 A kind of live streaming exchange method, system, device and electronic equipment
CN111050185B (en) * 2018-10-15 2022-08-16 武汉斗鱼网络科技有限公司 Live broadcast room wheat-connected video mixing method, storage medium, electronic equipment and system
CN109727501A (en) * 2019-01-07 2019-05-07 北京汉博信息技术有限公司 A kind of tutoring system
CN111866440B (en) * 2020-07-29 2021-03-09 全时云商务服务股份有限公司 Method, device and equipment for pushing video data and storage medium
CN113225577B (en) * 2021-05-07 2022-10-11 北京百度网讯科技有限公司 Live stream processing method, device and system, electronic equipment and storage medium
CN117376590A (en) * 2022-06-30 2024-01-09 北京字跳网络技术有限公司 View rendering method, view rendering device, electronic device, storage medium and program product

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005341325A (en) * 2004-05-28 2005-12-08 Nippon Telegr & Teleph Corp <Ntt> Multi-point video conference system, multi-point video conference control method, server apparatus, multi-point video conference control program, and program recording medium thereof
US20140063177A1 (en) * 2012-09-04 2014-03-06 Cisco Technology, Inc. Generating and Rendering Synthesized Views with Multiple Video Streams in Telepresence Video Conference Sessions
CN104486339A (en) * 2014-12-16 2015-04-01 广州华多网络科技有限公司 Method and device for displaying recommendation data in social application
CN105306468A (en) * 2015-10-30 2016-02-03 广州华多网络科技有限公司 Method for real-time sharing of synthetic video data and anchor client side
CN105357542A (en) * 2015-11-20 2016-02-24 广州华多网络科技有限公司 Live broadcast method, device and system
CN105828091A (en) * 2016-03-28 2016-08-03 广州华多网络科技有限公司 Method and system for video program playing in network broadcast
CN106658205A (en) * 2016-11-22 2017-05-10 广州华多网络科技有限公司 Studio video streaming synthesis control method, device and terminal equipment

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106162221A (en) * 2015-03-23 2016-11-23 阿里巴巴集团控股有限公司 The synthetic method of live video, Apparatus and system
CN105872569A (en) * 2015-11-27 2016-08-17 乐视云计算有限公司 Video playing method and system, and devices
CN106028137A (en) * 2016-06-22 2016-10-12 北京小米移动软件有限公司 Live streaming processing method and apparatus

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005341325A (en) * 2004-05-28 2005-12-08 Nippon Telegr & Teleph Corp <Ntt> Multi-point video conference system, multi-point video conference control method, server apparatus, multi-point video conference control program, and program recording medium thereof
US20140063177A1 (en) * 2012-09-04 2014-03-06 Cisco Technology, Inc. Generating and Rendering Synthesized Views with Multiple Video Streams in Telepresence Video Conference Sessions
CN104486339A (en) * 2014-12-16 2015-04-01 广州华多网络科技有限公司 Method and device for displaying recommendation data in social application
CN105306468A (en) * 2015-10-30 2016-02-03 广州华多网络科技有限公司 Method for real-time sharing of synthetic video data and anchor client side
CN105357542A (en) * 2015-11-20 2016-02-24 广州华多网络科技有限公司 Live broadcast method, device and system
CN105828091A (en) * 2016-03-28 2016-08-03 广州华多网络科技有限公司 Method and system for video program playing in network broadcast
CN106658205A (en) * 2016-11-22 2017-05-10 广州华多网络科技有限公司 Studio video streaming synthesis control method, device and terminal equipment

Cited By (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111182348A (en) * 2018-11-09 2020-05-19 阿里巴巴集团控股有限公司 Live broadcast picture display method and device
CN111182348B (en) * 2018-11-09 2022-06-14 阿里巴巴集团控股有限公司 Live broadcast picture display method and device, storage device and terminal
CN112312144B (en) * 2019-07-31 2022-11-29 腾讯科技（深圳）有限公司 Live broadcast method, device, equipment and storage medium
CN112312144A (en) * 2019-07-31 2021-02-02 腾讯科技（深圳）有限公司 Live broadcast method, device, equipment and storage medium
CN110740346B (en) * 2019-10-23 2022-04-22 北京达佳互联信息技术有限公司 Video data processing method, device, server, terminal and storage medium
CN110740346A (en) * 2019-10-23 2020-01-31 北京达佳互联信息技术有限公司 Video data processing method, device, server, terminal and storage medium
CN113163216B (en) * 2020-01-22 2023-08-22 华为云计算技术有限公司 Live broadcast method and live broadcast system based on cloud mobile phone
CN113163216A (en) * 2020-01-22 2021-07-23 华为技术有限公司 Live broadcast method and live broadcast system based on cloud mobile phone
US11882336B2 (en) 2020-02-11 2024-01-23 Shanghai Bilibili Technology Co., Ltd. Method and system for interaction in live streaming
CN113259692A (en) * 2020-02-11 2021-08-13 上海哔哩哔哩科技有限公司 Live broadcast interaction method and system
CN112291502A (en) * 2020-02-24 2021-01-29 北京字节跳动网络技术有限公司 Information interaction method, device and system and electronic equipment
CN113727164B (en) * 2020-05-26 2024-04-26 百度在线网络技术（北京）有限公司 Live broadcasting room entrance display method and device, electronic equipment and storage medium
CN113727164A (en) * 2020-05-26 2021-11-30 百度在线网络技术（北京）有限公司 Live broadcast room entrance display method and device, electronic equipment and storage medium
CN111954054A (en) * 2020-06-05 2020-11-17 筑觉绘(上海)科技有限公司 Image processing method, system, storage medium and computer device
CN111814732B (en) * 2020-07-23 2024-02-09 度小满科技(北京)有限公司 Identity verification method and device
CN111814732A (en) * 2020-07-23 2020-10-23 上海优扬新媒信息技术有限公司 Identity verification method and device
CN112272283A (en) * 2020-09-14 2021-01-26 重庆国翔创新教学设备有限公司 Multi-party online learning method based on double cameras, client and server
CN112188222A (en) * 2020-09-25 2021-01-05 吕庆新 Information acquisition processing device and application thereof, information transfer method, computer readable medium, and information acquisition processing method
CN114765695A (en) * 2021-01-15 2022-07-19 北京字节跳动网络技术有限公司 Live broadcast data processing method, device, equipment and medium
CN112954370B (en) * 2021-01-28 2023-09-26 深圳点猫科技有限公司 Encoding method, device and equipment for audio and video live broadcast
CN112954370A (en) * 2021-01-28 2021-06-11 深圳点猫科技有限公司 Encoding method, device and equipment for audio and video live broadcast
EP4220368A4 (en) * 2021-05-13 2024-03-13 Tencent Tech Shenzhen Co Ltd Multimedia data processing method and apparatus, and device, computer-readable storage medium and computer program product
CN113453033A (en) * 2021-06-29 2021-09-28 广州方硅信息技术有限公司 Live broadcast room information transmission processing method and device, equipment and medium thereof
CN113453033B (en) * 2021-06-29 2023-01-20 广州方硅信息技术有限公司 Live broadcasting room information transmission processing method and device, equipment and medium thereof
CN113556610B (en) * 2021-07-06 2023-07-28 广州方硅信息技术有限公司 Video synthesis control method and device, equipment and medium thereof
CN113556610A (en) * 2021-07-06 2021-10-26 广州方硅信息技术有限公司 Video synthesis control method, device, apparatus and medium
CN114051170A (en) * 2021-11-16 2022-02-15 北京达佳互联信息技术有限公司 Live broadcast processing method and device, electronic equipment and computer readable storage medium
CN114466207A (en) * 2022-01-18 2022-05-10 阿里巴巴（中国）有限公司 Live broadcast control method and computer storage medium
CN116708867B (en) * 2023-08-07 2023-11-10 深圳市亿晟科技有限公司 Live broadcast data processing method, device, equipment and storage medium
CN116708867A (en) * 2023-08-07 2023-09-05 深圳市亿晟科技有限公司 Live broadcast data processing method, device, equipment and storage medium

Also Published As

Publication number Publication date
CN106658205B (en) 2020-09-04
CN106658205A (en) 2017-05-10

Similar Documents

Publication Publication Date Title
WO2018095174A1 (en) Control method, device, and terminal apparatus for synthesizing video stream of live streaming room
WO2018095146A1 (en) Method, device, and terminal apparatus for synthesizing video stream of live streaming room
CN110597774B (en) File sharing method, system, device, computing equipment and terminal equipment
US10579243B2 (en) Theming for virtual collaboration
US11606597B2 (en) Devices, systems, and processes for facilitating live and recorded content watch parties
US20170374410A1 (en) Information processing apparatus, information processing method, and program
WO2018121014A1 (en) Video play control method and apparatus and terminal device
US9584835B2 (en) System and method for broadcasting interactive content
WO2018094556A1 (en) Live-broadcasting room video stream pushing control method, and corresponding server and mobile terminal
EP2933982B1 (en) Media stream transfer method and user equipment
US20120017149A1 (en) Video whisper sessions during online collaborative computing sessions
WO2015117513A1 (en) Video conference control method and system
CN106448297A (en) Cloud audio-video remote interactive class system
US9756096B1 (en) Methods for dynamically transmitting screen images to a remote device
KR20140103156A (en) System, apparatus and method for utilizing a multimedia service
WO2015003532A1 (en) Multimedia conferencing establishment method, device and system
CN112788276A (en) Video stream display method, transmission method, device, terminal, server and medium
US20230199239A1 (en) Streaming video trunking
WO2014177082A1 (en) Video conference video processing method and terminal
WO2017185709A1 (en) Television resource sharing method and apparatus, and television terminal
CN113727177B (en) Screen-throwing resource playing method and device, equipment and medium thereof
CN111385590A (en) Live broadcast data processing method and device and terminal
CN113726534A (en) Conference control method, conference control device, electronic equipment and storage medium
KR102149004B1 (en) Method and apparatus for generating multi channel images using mobile terminal
CN113573117A (en) Video live broadcast method and device and computer equipment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17873215

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17873215

Country of ref document: EP

Kind code of ref document: A1