CN116248950A - Interactive video multi-picture presentation method and device, storage medium and terminal - Google Patents

Interactive video multi-picture presentation method and device, storage medium and terminal Download PDF

Info

Publication number
CN116248950A
CN116248950A CN202310028674.1A CN202310028674A CN116248950A CN 116248950 A CN116248950 A CN 116248950A CN 202310028674 A CN202310028674 A CN 202310028674A CN 116248950 A CN116248950 A CN 116248950A
Authority
CN
China
Prior art keywords
picture
video
video data
layout
data stream
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310028674.1A
Other languages
Chinese (zh)
Inventor
唐高望
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Miluoxing Technology Group Co ltd
Original Assignee
Hangzhou Miluoxing Technology Group Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Miluoxing Technology Group Co ltd filed Critical Hangzhou Miluoxing Technology Group Co ltd
Priority to CN202310028674.1A priority Critical patent/CN116248950A/en
Publication of CN116248950A publication Critical patent/CN116248950A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235—Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip

Abstract

The invention discloses an interactive video multi-picture presentation method and device, a storage medium and a terminal, wherein the method comprises the following steps: acquiring multiple paths of videos to be combined, and decoding the videos to be combined into corresponding video data streams to be combined; combining all the video data streams to be combined into a composite video data stream according to a conventional layout mode, obtaining picture layout information of the composite video data stream, generating a composite picture video based on the composite video data stream and the picture layout information, pushing the composite picture video to a cloud server platform, so that after a viewing end obtains the composite picture video from the cloud server platform, the picture display layout of the composite video data stream can be adjusted based on the picture layout information; wherein the picture layout information includes position information of each path of video pictures in a picture display of the composite video data stream. On the basis that only one synthesized picture video is needed to be synthesized, a plurality of picture presentation modes are provided for the user, the number of the combined stream pushing is reduced, the occupation of server resources is greatly reduced, and the cost is saved.

Description

Interactive video multi-picture presentation method and device, storage medium and terminal
Technical Field
The present invention relates to the field of live broadcast image display technologies, and in particular, to an interactive video multi-image presentation method and apparatus, a storage medium, and a terminal.
Background
When watching live programs or surfing the internet, multi-person interaction scenes, such as a two-person PK scene, a multi-person warfare scene, a spectator's wheat scene (student wheat surfing and teacher interaction) and the like, are frequently encountered. In the interactive scene, interactive videos among interactive users are realized by mutually sliding by adopting an RTC technology. And the interactive pictures seen by the common audience are videos converged by the service end. The current service end confluence mode is as follows: pulling video data streams of all interactive users and decoding to obtain pictures of all users; carrying out picture combining processing on pictures of each user according to the appointed layout requirement, and encoding to obtain a new video stream; and the common audience user side pulls the combined video stream from the server for watching.
In order to meet the requirement that users see different picture layouts, the server needs to perform multiple picture combining processing, namely, the server needs to synthesize a new video for each picture layout, so that a large amount of CPU resources are occupied, and the fluency of live broadcasting is slowed down. And when the multi-user interaction picture exists, because the CPU resource is limited, only limited picture arrangement display modes can exist, and the user experience is reduced.
Disclosure of Invention
The invention aims to solve the technical problem that in the current interactive video generation process, in order to meet the requirements of users for seeing different picture layouts, a server needs to perform multiple picture combining processing, occupies a large amount of CPU resources, and drags live broadcast fluency.
In order to solve the above technical problems, the present invention provides an interactive video multi-picture presentation method, including:
acquiring multiple paths of videos to be combined, and decoding the videos to be combined into corresponding video data streams to be combined;
combining all the video data streams to be combined into a composite video data stream according to a conventional layout mode, acquiring picture layout information of the composite video data stream, generating a composite picture video based on the composite video data stream and the picture layout information, pushing the composite picture video to a cloud server platform, so that a viewing end can adjust picture display layout of the composite video data stream based on the picture layout information after acquiring the composite picture video from the cloud server platform;
wherein the picture layout information includes position information of each path of video pictures in picture display of the composite video data stream.
Preferably, the videos to be combined are videos which are located in the same interaction channel and are related to interaction.
Preferably, the interactive video is a live video or a net lesson video.
Preferably, generating the composite picture video based on the composite video data stream and the picture layout information comprises:
the picture layout information is added to the composite video data stream to generate a composite video data stream having picture layout information, and the composite video data stream having picture layout information is encoded to generate a composite picture video.
Preferably, the adjusting, by the viewing end, the picture display layout of the composite video data stream based on the picture layout information includes:
selecting a picture layout mode from the picture layout database as a picture layout mode to be executed;
when the picture layout mode to be executed is the same as the conventional layout mode, not adjusting the picture display layout of the synthesized video data stream;
and when the picture layout mode to be executed is different from the conventional layout mode, adjusting the picture display layout of the synthesized video data stream to the picture layout mode to be executed based on the picture layout information.
Preferably, adjusting the picture display layout of the composite video data stream to the picture layout mode to be performed based on the picture layout information includes:
and reading each path of video pictures from each frame of image data of the synthesized video data stream based on the position information of each path of video pictures in the picture layout information, and rearranging each path of video pictures in each frame of image data based on the picture layout mode to be executed so that the synthesized video data stream formed by rearranged image data is displayed in the picture layout mode to be executed.
Preferably, the picture layout database includes a plurality of picture-in-picture landscape layout modes in which different video pictures are used as main expression pictures, a plurality of picture-in-picture portrait layout modes in which different video pictures are used as main expression pictures, a plurality of landscape layout modes in which video pictures are arranged in different orders, a plurality of portrait layout modes in which video pictures are arranged in different orders, and a hidden layout mode of different video pictures.
In order to solve the technical problem, the invention also provides an interactive video multi-picture presentation device which comprises a video acquisition module and a synthesized picture video generation module;
the video acquisition module is used for acquiring multiple paths of videos to be combined and decoding the videos to be combined into corresponding video data streams to be combined;
the synthesized picture video generation module is used for combining all the video data streams to be combined into a synthesized video data stream according to a conventional layout mode, obtaining picture layout information of the synthesized video data stream, generating synthesized picture videos based on the synthesized video data stream and the picture layout information, pushing the synthesized picture videos to a cloud server platform, and adjusting picture display layout of the synthesized video data stream based on the picture layout information after a viewing end obtains the synthesized picture videos from the cloud server platform;
wherein the picture layout information includes position information of each path of video pictures in picture display of the composite video data stream.
In order to solve the above technical problem, the present invention further provides a storage medium, on which a computer program is stored, which when executed by a processor, implements the interactive video multi-picture presentation method.
In order to solve the technical problem, the present invention further provides a terminal, including: the device comprises a processor and a memory, wherein the memory is in communication connection with the processor;
the memory is used for storing a computer program, and the processor is used for executing the computer program stored in the memory so as to enable the terminal to execute the interactive video multi-picture presentation method.
One or more embodiments of the above-described solution may have the following advantages or benefits compared to the prior art:
by the method for presenting the multiple pictures of the interactive video, provided by the embodiment of the invention, the picture layout information of the synthesized video data stream is acquired, and the picture layout information is set in the synthesized picture video, so that a user at a watching end can adjust the picture display layout of the synthesized video data stream based on the picture layout information after acquiring the synthesized picture video, and further the interactive video with different picture layouts is obtained. The interactive video multi-picture presentation method provides a plurality of picture presentation modes for users on the basis that only one synthesized picture video is needed to be synthesized, reduces the number of combined stream pushing, greatly reduces the occupation of server resources and saves the cost.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention, without limitation to the invention. In the drawings:
FIG. 1 is a flow chart illustrating an interactive video multi-frame rendering method according to an embodiment of the invention;
fig. 2 shows a schematic diagram of data transmission of a live PK example according to the first embodiment of the present invention;
fig. 3 is a schematic diagram illustrating data transmission of a class example in the first embodiment of the present invention;
fig. 4 is a schematic structural diagram of an interactive video multi-screen presentation device according to a second embodiment of the present invention;
fig. 5 shows a schematic structural diagram of a fourth terminal according to an embodiment of the present invention.
Detailed Description
The following will describe embodiments of the present invention in detail with reference to the drawings and examples, thereby solving the technical problems by applying technical means to the present invention, and realizing the technical effects can be fully understood and implemented accordingly. It should be noted that, as long as no conflict is formed, each embodiment of the present invention and each feature of each embodiment may be combined with each other, and the formed technical solutions are all within the protection scope of the present invention.
RTC (Real time communication) real-time communication is an abbreviation of real-time audio and video, and commonly referred to as WebRTC technology, has been released as a formal standard by W3C and IETF.
The cloud server platform is CDN (Content Delivery Network), is a content delivery network, and aims to deliver the content of the network to the network 'edge' closest to a user through adding a new layer of network architecture into the existing Internet, so that the user can accelerate the CDN to acquire the required content and improve the response speed of the user for accessing the website.
Example 1
In order to solve the technical problems in the prior art, the embodiment of the invention provides an interactive video multi-picture presentation method.
FIG. 1 is a flow chart illustrating an interactive video multi-frame rendering method according to an embodiment of the invention; referring to fig. 1, the method for presenting multiple pictures of an interactive video according to an embodiment of the present invention includes the following steps.
Step S101, obtaining multiple paths of videos to be combined, and decoding the videos to be combined into corresponding video data streams to be combined.
Specifically, in live interactive video or in online class interactive video, a plurality of video data exist in the same interactive channel. For example, when a live two-person PK scene, there would be video data of anchor a and video data of anchor B; in the interaction process of the net lessons, teachers share video data, students connect with wheat video data and the like. Further, in the interaction process, all interaction ends participating in interaction in the same interaction channel transmit own interaction videos to the RTC platform, and after the RTC platform acquires all interaction videos participating in interaction in the same interaction channel, all the interaction videos participating in interaction are used as videos to be combined. And then decoding each path of video to be combined based on the decoding parameters to obtain a video data stream to be combined corresponding to each path of video to be combined. All the acquired video data streams to be combined are used for generating a composite video data stream subsequently, so that the watching end can watch the interactive video by acquiring the composite picture video. The live broadcast interactive video comprises a two-person PK interactive scene, a multi-person warfare interactive scene and a spectator continuous wheat interactive scene; the net lesson interaction scene comprises a scene that a teacher interacts with at least one student.
In the same interactive channel, because all interactive terminals participating in the interaction display real-time videos, the interactive video of each interactive terminal needs to be transmitted to other interactive terminals in the same interactive channel through the RTC platform. For example, in a live two-person PK scene, video data of a anchor a needs to be transmitted to an anchor B through an RTC platform, so that a terminal where the anchor a is located can display video of the anchor a and video of the anchor B simultaneously; similarly, video data of the anchor B is also required to be transmitted to the anchor a through the RTC platform, so that the terminal where the anchor B is located can display the video of the anchor B and the video of the anchor a simultaneously.
Step 102, merging all the video data streams to be merged into a composite video data stream according to a conventional layout mode, obtaining picture layout information of the composite video data stream, generating a composite picture video based on the composite video data stream and the picture layout information, and pushing the composite picture video to a cloud server platform so that after a viewing end obtains the composite picture video from the cloud server platform, the picture display layout of the composite video data stream can be adjusted based on the picture layout information.
Specifically, after all the video data streams to be combined in the same interactive channel are acquired by the RTC platform, the merging processing is performed on all the video data streams to be combined according to a preset conventional layout mode, so as to acquire a composite video data stream. The conventional layout manner may be configured to layout the video corresponding to the initiating terminal at the main display position, and order the other videos in the order of joining the interactive scenes. For example, in a two-person PK interaction scenario, setting the left video display bit to the primary display position; in the interaction scene of the multi-people warfare, taking the central video display position or the first video display position as a main display position; in the audience wheat-linked interaction scene, taking the maximum video display position as a main display position; and taking the maximum video display position as a main display position in the net lesson interaction scene. The main display position can also be set based on the actual situation, and is not excessively limited here.
Since all video data streams include multi-frame image data, the process of merging all video data streams to be merged is essentially: and merging the multi-frame image data of the multiple video data streams to be merged in the same time period into one frame of image data according to a conventional layout mode, and finally forming a composite video data stream by all merged image data.
After the composite video data stream is acquired, the picture layout information of the composite video data stream is also acquired, and further, the position information of each path of video picture in the picture display of the composite video data stream is acquired. After the picture layout information is acquired, a corresponding synthesized picture video can be generated based on the synthesized video data stream and the picture layout information. The picture layout information may further be added to the composite video data stream, for example, the picture layout information may be added to an SEI field of the H264 data to generate the composite video data stream with the picture layout information; and then encoding the composite video data stream with the picture layout information based on the encoding parameters to generate a composite picture video, so that the picture layout information of the composite video data stream can be transmitted to a viewing end of a spectator along with the transmission of the composite picture video. Another way to obtain the composite picture video is: firstly, coding a synthesized video data stream through coding parameters to obtain a temporary synthesized picture video; the picture layout information is then added to the temporary composite picture video to obtain the temporary composite picture video (i.e., composite picture video) having the picture layout information.
It should be noted that, the picture layout information and the layout manner are different, the picture layout information refers to specific position information of each video picture, for example, in a two-person PK interaction scene, specific position information of the anchor a video is a [ x=0, y= 0,w =240, h=360 ], and the layout manner is layout information of the video picture in the whole image data, for example, an up-down layout, a left-right layout, a picture-in-picture layout, a sequential layout, or the like.
And after the RTC platform acquires the composite picture video, pushing the composite picture video to the cloud server platform so that the viewing end can pull the composite picture video from the cloud server platform for rendering and playing. Further, after the viewing end obtains the synthesized picture video from the cloud server platform, the synthesized picture video is decoded, and the synthesized video data stream and the corresponding picture layout information are obtained. After the viewing end obtains the synthesized video data stream and the corresponding picture layout information, the picture display layout of the synthesized video data stream can be adjusted based on the picture layout mode in the picture layout database.
The adjusting of the picture display layout of the composite video data stream by the specific viewing end based on the picture layout information comprises: the user selects a proper picture layout mode from a picture layout database as a picture layout mode to be executed based on the equipment state, personal watching preference or requirement; after the viewing end obtains the picture layout mode to be executed of the user, judging whether the picture layout information of the currently obtained composite video data stream is the same as the picture layout mode to be executed, if so, directly displaying the composite video data stream, otherwise, adjusting the picture display layout of the composite video data stream based on the picture layout information, so that the adjusted composite video data stream is subjected to picture display in the picture layout mode to be executed.
Still further, adjusting the picture display layout of the composite video data stream to a picture layout mode to be performed based on the picture layout information includes: and reading each path of video pictures from each frame of image data of the synthesized video data stream based on the position information of each path of video pictures in the picture layout information, and re-laying each path of video pictures in each frame of image data based on a picture layout mode to be executed so that the synthesized video data stream formed by the re-laid image data can be displayed in the picture layout mode to be executed.
Preferably, the screen layout database includes a plurality of picture-in-picture landscape layout modes in which different video screens are used as main expression screens, a plurality of picture-in-picture portrait layout modes in which different video screens are used as main expression screens, a plurality of landscape layout modes in which video screens are arranged in different orders, a plurality of portrait layout modes in which video screens are arranged in different orders, a hidden layout mode of different video screens, and the like.
In order to better explain the interactive video multi-picture presentation method of the invention, the live two-person PK scene and the net lesson student interaction scene are taken as examples for respectively describing in detail.
Fig. 2 shows a schematic diagram of data transmission of a live PK example according to the first embodiment of the present invention; referring to fig. 2, assume that a presenter a has a viewer a1 watching and a presenter B has a viewer B1 watching, for live PK interactions. The logic flow related to the video in the scene is as follows:
anchor a: the camera collects videos and displays the videos on the left side of the PK interface, and the right side of the PK interface pulls and displays the video of the anchor B. And simultaneously, the anchor A pushes the acquired pictures to the RTC platform.
Anchor B: the camera collects videos and displays the videos on the left side of the PK interface, and the right side of the PK interface pulls and displays the video of the anchor A. And simultaneously, the anchor B pushes the acquired pictures to the RTC platform.
RTC platform confluence: and decoding videos of the anchor A and the anchor B to obtain video data streams of the anchor A and the anchor B, carrying out mapping according to a conventional layout mode (an anchor A picture is on the left side and an anchor B picture is on the right side), obtaining a synthesized video data stream, obtaining picture layout information of the synthesized video data stream, adding the picture layout information into the synthesized video data stream to generate a synthesized picture video, and finally pushing the synthesized picture video to a cloud server platform.
And the audience a1 pulls the synthesized picture video from the cloud server platform, analyzes picture layout information in the synthesized picture video, and directly renders and plays the synthesized video data stream in accordance with the service requirement in the conventional layout mode.
And the audience b1 pulls the synthesized picture video from the cloud server platform, analyzes picture layout information in the synthesized picture video, exchanges left and right pictures in the synthesized video data stream according to service requirements, and then renders and plays the exchanged synthesized video data stream.
Fig. 3 is a schematic diagram illustrating data transmission of a class example in the first embodiment of the present invention; referring to fig. 3, assume that a teacher is a lecturer of a net class, a student S1 and a student S2 are in a student-on-the-fly state, and a student S3-student S5 is watching. The logic flow related to the video in the scene is as follows:
teacher T: the PC side pushes courseware lecture pictures to the RTC platform, and simultaneously pulls and displays videos of the students S1 and S2 on the wheat.
The student S1 on the wheat pushes own pictures to the RTC platform by the mobile phone end, and simultaneously pulls and displays videos of teachers and students S2.
And the student S2 on the wheat pushes own pictures to the RTC platform by the mobile phone end, and simultaneously pulls and displays videos of teachers and students S1.
RTC platform confluence: and decoding videos of teachers and wheat students, carrying out picture combining according to a conventional layout mode (the teacher T is a picture-in-picture large picture, the wheat students S1 and the wheat students S2 are picture-in-picture small pictures), obtaining a composite video data stream, obtaining picture layout information of the composite video data stream, adding the picture layout information into the composite video data stream to generate a composite picture video, and finally pushing the composite picture video to a cloud server platform.
And the student S3 under wheat pulls the synthesized picture video from the cloud server platform, analyzes picture layout information in the synthesized picture video, selects a certain picture-in-picture transverse layout mode from a picture layout database, adjusts picture display layout in the video data stream based on the selected picture-in-picture transverse layout mode, and renders and plays the adjusted synthesized video data stream.
And S4, pulling the synthesized picture video from the cloud server platform, analyzing picture layout information in the synthesized picture video, selecting a certain longitudinal layout mode which is arranged in sequence from a picture layout database, adjusting picture display layout in the video data stream based on the selected longitudinal layout mode, and rendering and playing the adjusted synthesized video data stream.
And the student S5 under wheat pulls the synthesized picture video from the cloud server platform, analyzes picture layout information in the synthesized picture video, selects a certain hidden layout mode from a picture layout database, adjusts picture display layout in the video data stream based on the selected hidden layout mode to hide video pictures of the student S1 under wheat and the student S2 under wheat, and finally only renders and plays courseware lecture pictures.
According to the interactive video multi-picture presentation method provided by the embodiment of the invention, the picture layout information of the synthesized video data stream is acquired, and the picture layout information is set in the synthesized picture video, so that a user at a watching end can adjust the picture display layout of the synthesized video data stream based on the picture layout information after acquiring the synthesized picture video, and further the interactive video with different picture layouts is obtained. The interactive video multi-picture presentation method provides a plurality of picture presentation modes for users on the basis that only one synthesized picture video is needed to be synthesized, reduces the number of combined stream pushing, greatly reduces the occupation of server resources and saves the cost.
Example two
In order to solve the technical problems in the prior art, the embodiment of the invention provides an interactive video multi-picture presentation device.
Fig. 4 is a schematic structural diagram of an interactive video multi-screen presentation device according to a second embodiment of the present invention; referring to fig. 4, the interactive video multi-picture presentation apparatus according to the embodiment of the present invention includes a video acquisition module and a composite picture video generation module.
The video acquisition module is used for acquiring multiple paths of videos to be combined and decoding the videos to be combined into corresponding video data streams to be combined.
The synthesized picture video generation module is used for combining all the video data streams to be combined into a synthesized video data stream according to a conventional layout mode, obtaining picture layout information of the synthesized video data stream, generating synthesized picture videos based on the synthesized video data stream and the picture layout information, pushing the synthesized picture videos to the cloud server platform, and adjusting picture display layout of the synthesized video data stream based on the picture layout information after a viewing end obtains the synthesized picture videos from the cloud server platform.
Wherein the picture layout information includes position information of each path of video pictures in the picture display of the composite video data stream.
According to the interactive video multi-picture presentation device provided by the embodiment of the invention, the picture layout information of the synthesized video data stream is acquired, and the picture layout information is set in the synthesized picture video, so that a user at a watching end can adjust the picture display layout of the synthesized video data stream based on the picture layout information after acquiring the synthesized picture video, and further, interactive videos with different picture layouts are obtained. The interactive video multi-picture presentation device provides multiple picture presentation modes for users on the basis that only one synthesized picture video is needed to be synthesized, reduces the number of combined stream pushing, greatly reduces the occupation of server resources and saves the cost.
Example III
In order to solve the above-mentioned technical problems in the prior art, an embodiment of the present invention further provides a storage medium storing a computer program, where the computer program can implement all the steps in the interactive video multi-picture presentation method according to the first embodiment when executed by a processor.
The specific steps of the interactive video multi-picture presentation method and the beneficial effects obtained by applying the readable storage medium provided by the embodiment of the invention are the same as those of the first embodiment, and are not described in detail herein.
It should be noted that: the storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.
Example IV
In order to solve the technical problems in the prior art, the embodiment of the invention also provides a terminal.
Fig. 5 shows a schematic diagram of a fourth terminal structure according to an embodiment of the present invention, and referring to fig. 5, the terminal of the present embodiment includes a processor and a memory that are connected to each other; the memory is used for storing a computer program, and the processor is used for executing the computer program stored in the memory, so that the terminal can realize all the steps in the interactive video multi-picture presentation method in the first embodiment when executing the computer program.
The specific steps of the interactive video multi-picture presentation method and the beneficial effects obtained by the terminal provided by the embodiment of the invention are the same as those of the first embodiment, and are not repeated here.
It should be noted that the memory may include a random access memory (Random Access Memory, abbreviated as RAM) and may further include a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. The same processor may be a general processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field programmable gate arrays (Field Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
Although the embodiments of the present invention are disclosed above, the embodiments are only used for the convenience of understanding the present invention, and are not intended to limit the present invention. Any person skilled in the art can make any modification and variation in form and detail without departing from the spirit and scope of the present disclosure, but the scope of the present disclosure is still subject to the scope of the present disclosure as defined by the appended claims.

Claims (10)

1. An interactive video multi-picture presentation method, comprising:
acquiring multiple paths of videos to be combined, and decoding the videos to be combined into corresponding video data streams to be combined;
combining all the video data streams to be combined into a composite video data stream according to a conventional layout mode, acquiring picture layout information of the composite video data stream, generating a composite picture video based on the composite video data stream and the picture layout information, and pushing the composite picture video to a cloud server platform so that a viewing end can adjust picture display layout of the composite video data stream based on the picture layout information after acquiring the composite picture video from the cloud server platform;
wherein the picture layout information includes position information of each path of video pictures in picture display of the composite video data stream.
2. The method of claim 1, wherein the videos to be combined are all videos that are related to interaction in the same interaction channel.
3. The method of claim 1, wherein the interactive video is live video or net lesson video.
4. The method of claim 1, wherein generating a composite picture video based on the composite video data stream and the picture layout information comprises:
the picture layout information is added to the composite video data stream to generate a composite video data stream having picture layout information, and the composite video data stream having picture layout information is encoded to generate a composite picture video.
5. The method of claim 1, wherein the adjusting of the picture display layout of the composite video data stream by the viewing end based on the picture layout information comprises:
selecting a picture layout mode from the picture layout database as a picture layout mode to be executed;
when the picture layout mode to be executed is the same as the conventional layout mode, not adjusting the picture display layout of the synthesized video data stream;
and when the picture layout mode to be executed is different from the conventional layout mode, adjusting the picture display layout of the synthesized video data stream to the picture layout mode to be executed based on the picture layout information.
6. The method of claim 5, wherein adjusting a picture display layout of the composite video data stream to the picture layout manner to be performed based on the picture layout information comprises:
and reading each path of video pictures from each frame of image data of the synthesized video data stream based on the position information of each path of video pictures in the picture layout information, and rearranging each path of video pictures in each frame of image data based on the picture layout mode to be executed so that the synthesized video data stream formed by rearranged image data is displayed in the picture layout mode to be executed.
7. The method of claim 5, wherein the screen layout database comprises a plurality of picture-in-picture landscape layout patterns with different video screens as main presentation screens, a plurality of picture-in-picture portrait layout patterns with different video screens as main presentation screens, a plurality of landscape layout patterns with video screens arranged in different orders, a plurality of portrait layout patterns with video screens arranged in different orders, and a hidden layout pattern with different video screens.
8. The interactive video multi-picture presentation device is characterized by comprising a video acquisition module and a synthesized picture video generation module;
the video acquisition module is used for acquiring multiple paths of videos to be combined and decoding the videos to be combined into corresponding video data streams to be combined;
the synthesized picture video generation module is used for combining all the video data streams to be combined into a synthesized video data stream according to a conventional layout mode, obtaining picture layout information of the synthesized video data stream, generating synthesized picture videos based on the synthesized video data stream and the picture layout information, pushing the synthesized picture videos to a cloud server platform, and adjusting picture display layout of the synthesized video data stream based on the picture layout information after a viewing end obtains the synthesized picture videos from the cloud server platform;
wherein the picture layout information includes position information of each path of video pictures in picture display of the composite video data stream.
9. A storage medium having stored thereon a computer program, which when executed by a processor implements the interactive video multi-picture presentation method of any one of claims 1 to 7.
10. A terminal, comprising: the device comprises a processor and a memory, wherein the memory is in communication connection with the processor;
the memory is used for storing a computer program, and the processor is used for executing the computer program stored in the memory, so that the terminal executes the interactive video multi-picture presentation method as claimed in any one of claims 1 to 7.
CN202310028674.1A 2023-01-09 2023-01-09 Interactive video multi-picture presentation method and device, storage medium and terminal Pending CN116248950A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310028674.1A CN116248950A (en) 2023-01-09 2023-01-09 Interactive video multi-picture presentation method and device, storage medium and terminal

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310028674.1A CN116248950A (en) 2023-01-09 2023-01-09 Interactive video multi-picture presentation method and device, storage medium and terminal

Publications (1)

Publication Number Publication Date
CN116248950A true CN116248950A (en) 2023-06-09

Family

ID=86634196

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310028674.1A Pending CN116248950A (en) 2023-01-09 2023-01-09 Interactive video multi-picture presentation method and device, storage medium and terminal

Country Status (1)

Country Link
CN (1) CN116248950A (en)

Similar Documents

Publication Publication Date Title
US10531158B2 (en) Multi-source video navigation
CN102905170B (en) Screen popping method and system for video
US7913156B2 (en) Rich media event production system and method including the capturing, indexing, and synchronizing of RGB-based graphic content
US20130173709A1 (en) Cloud-based content mixing into one stream
US20070150612A1 (en) Method and system of providing multimedia content
US20090287790A1 (en) System and Method for Providing a Virtual Environment with Shared Video on Demand
Zoric et al. Panoramic video: design challenges and implications for content interaction
KR101369273B1 (en) Interactive live broadcasting system and method
Barkhuus et al. Watching the footwork: Second screen interaction at a dance and music performance
CN113518232A (en) Video display method, device, equipment and storage medium
Curcio et al. 360-degree video streaming and its subjective quality
CN114461423A (en) Multimedia stream processing method, device, storage medium and program product
CN112312066A (en) Synchronous teaching technology based on multi-terminal real-time green screen synthesis and application
CN116248950A (en) Interactive video multi-picture presentation method and device, storage medium and terminal
JP7408506B2 (en) Content distribution system, content distribution method, and content distribution program
US20080013917A1 (en) Information intermediation system
Richards The unofficial guide to open broadcaster software
CN1745584A (en) Method of and system for augmenting presentation of content
Lugmayr et al. E= MC2+ 1: a fully digital, collaborative, high-definition (HD) production from scene to screen
US11792380B2 (en) Video transmission method, video processing device, and video generating system for virtual reality
Koentjoro Exploring the ability of Virtual Set Systems in helping Small Broadcasters entering the Digital Television scene: Addressing the Technical, Licensing, and Operational Challenges of Fatwa TV
CN106713994A (en) Method and device for generating electronic calendar
Series Collection of usage scenarios of advanced immersive sensory media systems
Series Collection of usage scenarios and current statuses of advanced immersive audio-visual systems
Thompson Travails with My Camera: Investigations with Prototype Workflows and Specialist Cameras for Wider Target Platform Coverage, Reduced Complexity, and Universal Distribution

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination