CN117714801A - Multi-user audio/video interactive live broadcast data interaction method, equipment and storage medium - Google Patents
Multi-user audio/video interactive live broadcast data interaction method, equipment and storage medium Download PDFInfo
- Publication number
- CN117714801A CN117714801A CN202311727426.2A CN202311727426A CN117714801A CN 117714801 A CN117714801 A CN 117714801A CN 202311727426 A CN202311727426 A CN 202311727426A CN 117714801 A CN117714801 A CN 117714801A
- Authority
- CN
- China
- Prior art keywords
- audio
- terminal
- interactive
- video data
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 191
- 230000003993 interaction Effects 0.000 title claims abstract description 78
- 238000000034 method Methods 0.000 title claims abstract description 75
- 238000004590 computer program Methods 0.000 claims description 12
- 230000000977 initiatory effect Effects 0.000 claims description 8
- 230000009286 beneficial effect Effects 0.000 abstract description 15
- 230000000694 effects Effects 0.000 abstract description 9
- 238000012545 processing Methods 0.000 description 24
- 238000010586 diagram Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/233—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/2368—Multiplexing of audio and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4341—Demultiplexing of audio and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a multi-user audio/video interactive live broadcast data interaction method, electronic equipment and a computer readable storage medium. The data forwarding flow of the method comprises the following steps: receiving audio and video data acquired from an interactive terminal; the audio and video data collected from the interaction terminal are forwarded to the main interaction terminal; receiving mixed audio and video data generated by a main interactive terminal; and forwarding the mixed audio and video data generated by the master interactive terminal to the slave interactive terminal. The method is beneficial to reducing the performance requirement of the server or the configuration requirement of the terminal equipment, and is also beneficial to reducing the network bandwidth pressure, so that a good interactive live broadcast effect is ensured.
Description
Technical Field
The invention relates to the technical field of streaming media, in particular to a multi-user audio/video interactive live broadcast data interaction method, electronic equipment and a computer readable storage medium.
Background
With the development of streaming media and live broadcasting technology, multi-user audio and video interactive live broadcasting is also gradually becoming a popular live broadcasting mode. The terminal equipment participating in the interactive live broadcast can be called as an interactive participant or an interactive terminal of the interactive live broadcast, and is required to be responsible for acquisition, receiving, transmitting, decoding, playing and the like of audio and video data. On the other hand, there will be a terminal device on the network that does not participate in the interactive live broadcast, but decodes and plays the audio and video data of the interactive live broadcast, which may be called as a live viewer or a terminal for viewing the interactive live broadcast.
In the process of multi-user audio and video interactive live broadcast, audio and video data collected by all interactive terminals participating in the interactive live broadcast are required to be subjected to mixed processing, namely, multi-path audio and video data are subjected to mixed processing to generate one path of mixed audio and video data, and the mixed audio and video data are decoded and played by the interactive terminals and the watching terminals, so that the picture and sound of the interactive live broadcast are obtained.
Aiming at the data interaction of multi-user audio and video interaction live broadcast, the common processing modes are as follows:
1) And mixing the audio and video data by the server. Specifically, the server receives all audio and video data acquired by the interactive terminal, performs mixing processing to generate mixed audio and video data, and then sends the mixed audio and video data to the interactive terminal and the viewing terminal for decoding and playing. In the normal operation and maintenance process, a plurality of interactive live broadcast exist on the network at the same time, which means that the server needs to perform a plurality of audio and video mixed processing tasks at the same time, so that the server is required to have high processing performance, otherwise, the performance is easily reduced due to overhigh machine load, thereby influencing the interactive live broadcast effect and reducing the user experience.
2) The server only forwards the audio and video data collected by the interactive terminal, and the terminal equipment performs the mixing processing of the audio and video data. Specifically, the server receives audio and video data collected by all the interactive terminals and forwards the audio and video data to the terminal equipment needing to play the interactive live broadcast picture, the server does not do audio and video mixing processing, and the terminal equipment performs decoding, mixing, encoding and other mixing processing operations by itself, namely, each terminal equipment is required to have the decoding, mixing and encoding capability of simultaneously processing multiple paths of audio and video data, so that each terminal equipment has high configuration requirements, and the equipment cost of the interactive live broadcast is increased. In addition, the forwarding process of the audio and video data is very complicated, which also requires a high processing performance of the server and strong network bandwidth support.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide a multi-user audio/video interactive live broadcast data interaction method, electronic equipment and a computer readable storage medium, which are beneficial to reducing the performance requirement of a server or the configuration requirement of terminal equipment and are also beneficial to reducing the network bandwidth pressure so as to ensure a good interactive live broadcast effect.
The multi-user audio and video interactive live broadcast data interaction method is realized by adopting the following technical scheme:
the method for interacting the live broadcast data of the multi-user audio/video interaction comprises the following steps:
receiving a request instruction for initiating interactive live broadcast;
determining all terminal devices participating in the interactive live broadcast according to a request instruction, wherein one of the terminal devices is a master interactive terminal of the interactive live broadcast, and the other terminal devices are slave interactive terminals of the interactive live broadcast;
executing a data forwarding process for the interactive live broadcast, including:
receiving audio and video data acquired from an interactive terminal;
the audio and video data collected from the interaction terminal are forwarded to the main interaction terminal;
receiving mixed audio and video data generated by a main interactive terminal;
and forwarding the mixed audio and video data generated by the master interactive terminal to the slave interactive terminal.
Further, the data forwarding process further includes:
and pushing the mixed audio and video data generated by the main interactive terminal to a viewing terminal or a streaming media platform.
Further, the method further comprises:
allocating buffer areas for all terminal devices of the interactive live broadcast in a one-to-one correspondence manner;
the data forwarding flow further includes:
storing the audio and video data acquired from the interactive terminal into a buffer area corresponding to the interactive terminal;
when the audio and video data collected by the slave interaction terminal is forwarded to the master interaction terminal, the audio and video data are obtained from a buffer zone corresponding to the slave interaction terminal and forwarded to the master interaction terminal;
storing the mixed audio and video data generated by the main interactive terminal into a buffer area corresponding to the main interactive terminal;
when the mixed audio and video data generated by the master interactive terminal is forwarded to the slave interactive terminal, the mixed audio and video data is acquired from a buffer area corresponding to the master interactive terminal and forwarded to the slave interactive terminal.
Further, the buffer zone is a ring buffer zone.
The method for interacting the live broadcast data of the multi-user audio/video interaction comprises the following steps:
when the method is used as a main interaction terminal in interaction live broadcast, the method comprises the following steps:
collecting audio and video data;
receiving audio and video data forwarded by a server and collected by an interaction terminal;
mixing the audio and video data acquired by the local machine with the audio and video data acquired from the interactive terminal to generate mixed audio and video data;
uploading the generated mixed audio and video data to a server, and decoding and playing;
when the method is used as a slave interaction terminal in the interaction live broadcast, the method comprises the following steps:
collecting audio and video data;
uploading audio and video data acquired by a local machine to a server;
the method comprises the steps of receiving mixed audio and video data forwarded by a server and generated by a main interaction terminal;
and decoding and playing the received mixed audio and video data.
Further, the method further comprises:
when the method is used as a main interaction terminal in the interaction live broadcast, the method further comprises the following steps:
pushing the generated mixed audio and video data to a streaming media platform.
Further, the method further comprises:
selecting all terminal devices participating in the interactive live broadcast, wherein one of the terminal devices is a master interactive terminal of the interactive live broadcast, and the other terminal devices are slave interactive terminals of the interactive live broadcast;
and sending a request instruction for initiating the interactive live broadcast to a server.
Further, the mixing process includes:
separating each path of audio and video data into video data and audio data;
decoding each path of video data to obtain video decoding data, and decoding each path of audio data to obtain audio decoding data;
mixing each path of video decoding data to generate mixed video data, and mixing each path of audio decoding data to generate mixed audio data;
encoding the mixed video data to obtain mixed video encoded data, and encoding the mixed audio data to obtain mixed audio encoded data;
the mixed video encoded data and the mixed audio encoded data are combined to generate mixed audio-video data.
The electronic equipment is realized by adopting the following technical scheme:
the electronic equipment comprises a memory, a processor and a computer program which is stored in the memory and can run on the processor, wherein the processor realizes the multi-person audio/video interactive live broadcast data interaction method when executing the computer program.
The computer readable storage medium of the present invention is realized by the following technical scheme:
the computer readable storage medium is stored with a computer program, and when the computer program is executed by the processor, the multi-person audio/video interactive live broadcast data interaction method is realized.
Compared with the prior art, the invention has the beneficial effects that:
the multi-user audio/video interactive live broadcast data interaction method, the electronic equipment and the computer readable storage medium provided by the invention are beneficial to reducing the performance requirement of a server or the configuration requirement of terminal equipment, and are also beneficial to reducing the network bandwidth pressure, so that a good interactive live broadcast effect is ensured.
Drawings
Fig. 1 is a schematic diagram of a multi-user audio/video interactive live broadcast data interaction method according to an embodiment of the invention;
fig. 2 is a schematic illustration of a push flow of a multi-user audio/video interactive live broadcast data interaction method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a main interactive terminal according to an embodiment of the present invention;
fig. 4 is a flowchart of a method for interacting live broadcast data with multi-user audio and video according to an embodiment of the present invention.
Detailed Description
The present invention will be further described with reference to the accompanying drawings and detailed description, wherein it is to be understood that, on the premise of no conflict, the following embodiments or technical features may be arbitrarily combined to form new embodiments.
Referring to fig. 1-4, the embodiment of the invention provides a multi-user audio/video interactive live broadcast data interaction method, which comprises the steps of being applied to a server and being applied to terminal equipment.
The multi-user audio/video interactive live broadcast data interaction method is applied to a server and comprises steps S100-S400.
S100, receiving a request instruction for initiating interactive live broadcast.
S200, determining all terminal devices participating in the interactive live broadcast according to a request instruction, wherein in the step S200, one of the terminal devices is a master interactive terminal of the interactive live broadcast, and the rest is a slave interactive terminal of the interactive live broadcast.
S300, allocating buffer areas for all terminal equipment of interactive live broadcast in a one-to-one correspondence mode, wherein the buffer areas are specifically annular buffer areas, the annular buffer areas and the terminal equipment are in one-to-one correspondence, and for example, the buffer areas and the identification information (such as IP addresses or equipment identification codes) of the terminal equipment can be bound.
S400, executing a data forwarding flow for the interactive live broadcast, wherein the data forwarding flow comprises the following steps:
s401, receiving the audio and video data acquired from the interactive terminal, and storing the audio and video data acquired from the interactive terminal into a buffer area corresponding to the interactive terminal;
s402, forwarding the audio and video data acquired by the slave interaction terminal to the master interaction terminal, and when the step S402 is executed, specifically, acquiring the audio and video data from a buffer area corresponding to the slave interaction terminal and forwarding the audio and video data to the master interaction terminal;
s403, receiving the mixed audio and video data generated by the main interactive terminal, and storing the mixed audio and video data generated by the main interactive terminal into a buffer area corresponding to the main interactive terminal;
s404, forwarding the mixed audio and video data generated by the master interactive terminal to the slave interactive terminal, and when the step S404 is executed, specifically, acquiring the mixed audio and video data from a buffer area corresponding to the master interactive terminal and forwarding the mixed audio and video data to the slave interactive terminal;
s405, pushing the mixed audio and video data generated by the main interactive terminal to a viewing terminal or a streaming media platform; regarding step S405, the server pushes the mixed audio and video data generated by the main interactive terminal to the viewing terminal, which means that the server itself may also be used as a streaming media server to push the interactive live broadcast picture and sound to the viewing terminal; in addition, the server can push the mixed audio and video data generated by the main interactive terminal to the streaming media platform, and then the streaming media platform pushes the interactive live broadcast picture and sound to the watching terminal.
The multi-user audio/video interactive live broadcast data interaction method of the embodiment of the invention is applied to the steps of a server, can be executed by a single server or can be executed by a plurality of servers together, for example, one server can specially provide a data forwarding service for forwarding audio/video data of a single picture, and the other server can specially provide a streaming media service for pushing mixed audio/video data.
The multi-user audio/video interactive live broadcast data interaction method is applied to the steps of terminal equipment and comprises the steps C100-C400.
And C100, selecting all terminal devices participating in the interactive live broadcast, wherein in the step C100, one of the terminal devices is a master interactive terminal of the interactive live broadcast, and the rest is a slave interactive terminal of the interactive live broadcast.
And C200, sending a request instruction for initiating the interactive live broadcast to a server.
And C300, executing the following steps when the method is used as a main interactive terminal in the interactive live broadcast:
c301, collecting audio and video data;
c302, receiving audio and video data which are forwarded by a server and collected by a slave interaction terminal, wherein the slave interaction terminal and the slave interaction terminal in the step C302 participate in the same interactive live broadcast;
c303, mixing the audio and video data acquired by the local machine with the audio and video data acquired from the interactive terminal to generate mixed audio and video data;
c304, uploading the generated mixed audio and video data to a server, and decoding and playing;
c305, pushing the generated mixed audio and video data to a streaming media platform; the explanation of this step C305 is that the server is not required to push the mixed audio and video data, but the main interactive terminal itself pushes the mixed audio and video data to the streaming platform, and then the streaming platform pushes the interactive live broadcast picture and sound to the viewing terminal.
And C400, executing the following steps when the following steps are used as the slave interaction terminal in the interaction live broadcast:
c401, collecting audio and video data;
c402, uploading the audio and video data acquired by the local machine to a server;
c403, receiving the mixed audio and video data forwarded by the server and generated by the main interactive terminal, wherein the main interactive terminal in the step C403 participates in the same interactive live broadcast with the local;
and C404, decoding and playing the received mixed audio and video data.
In interactive live broadcast, a terminal device used as a master interactive terminal should possess decoding, mixing and encoding capabilities for simultaneously processing multiple audio and video data, and no related configuration is needed from the interactive terminal. In order to enable the main interactive terminal to realize the function of mixing and processing multiple paths of audio and video data, the main interactive terminal should be configured with a decapsulation unit, a video decoding unit, an audio decoding unit, a video mixing unit, an audio mixing unit, a video encoding unit, an audio encoding unit, and an audio and video combining unit.
In step C303, the mixing process includes:
separating each path of audio and video data (limited in the range of the local and the slave interaction terminals) into video data and audio data by a decapsulation unit;
decoding each path of video data through a video decoding unit to obtain video decoding data, and decoding each path of audio data through an audio decoding unit to obtain audio decoding data;
mixing each path of video decoding data through a video mixing unit to generate mixed video data, and mixing each path of audio decoding data through an audio mixing unit to generate mixed audio data;
the mixed video data is encoded through a video encoding unit to obtain mixed video encoded data, and the mixed audio data is encoded through an audio encoding unit to obtain mixed audio encoded data;
the mixed video encoded data and the mixed audio encoded data are combined by an audio-video composition unit, thereby generating mixed audio-video data.
Based on the above steps applied to the server and the steps applied to the terminal device, the method according to the embodiment of the present invention is illustrated with reference to fig. 1 to 3. In the interactive live broadcast 1, a total of N terminal devices participate in the interaction, and the overall flow of the interactive live broadcast 1 is described below with reference to fig. 4.
Step (1): the user initiating the interactive live broadcast operates on the terminal 1, selects all terminal devices (i.e., terminals 1 to N) participating in the interactive live broadcast 1 from a device list of a user interface of the terminal 1, and among the terminal devices, selects a local or another terminal device as a main interactive terminal, and sends a request instruction corresponding to the interactive live broadcast 1 to a server. Identification information, configuration information and the like of each terminal can be checked in the equipment list, and which terminal can be used as a main interaction terminal can be judged through the configuration information; the request instruction should carry the identification information of the terminals 1 to N. For example, the terminal initiating the interactive live broadcast and the main interactive terminal are both the terminal 1.
Step (2): the server receives a request instruction of the terminal 1, analyzes the request instruction to obtain the identification information of the terminals 1 to N, and conveniently binds the buffer area with the identification information of the terminals when the buffer area is allocated; and initiates an interaction request to the terminals 2-N.
Step (3): the terminals 2 to N respond to the request and formally enter a data forwarding flow.
Step (4): the audio and video data collected by each of the slave interaction terminals 2 to N are uploaded to a server, and the server forwards the audio and video data to the master interaction terminal 1.
Step (5): the main interaction terminal 1 receives the audio and video data collected by the terminals 2 to N, mixes the audio and video data collected by the main interaction terminal with the audio and video data collected by the terminals 2 to N to generate mixed audio and video data, and decodes and plays the mixed audio and video data.
Step (6): the main interactive terminal 1 uploads the mixed audio and video data to the server and pushes the mixed audio and video data to the streaming media platform, and then the streaming media platform pushes the interactive live broadcast pictures and sounds to the watching terminal.
Step (7): the server forwards the mixed audio and video data to the slave interactive terminals 2 to N.
According to the process, the multi-person audio/video interactive live broadcast data interaction method solves the data interaction problem of multi-person audio/video interactive live broadcast, and has the following advantages.
1) Compared with the mode of carrying out audio and video data mixed processing by a server in the prior art, the method has the advantages that: the mixed processing flow is distributed to the main interactive terminals participating in the interactive live broadcast, and the server mainly executes the data forwarding flow, so that the load of the server is greatly reduced, the performance requirement and the cost of the server are reduced, and the interactive live broadcast effect is ensured. In the prior art, if a plurality of interactive live broadcast exists on a network at the same time, a plurality of audio and video mixing tasks are concentrated on a server for processing, and the method distributes the audio and video mixing tasks to main interactive terminals for processing, and each main interactive terminal only needs to process the audio and video mixing tasks of the interactive live broadcast participated in by the main interactive terminal.
2) Compared with the mode that a server only forwards audio and video data and terminal equipment performs audio and video mixing processing in the prior art, the method has the advantages that: 2.1, in the prior art, as long as terminal equipment (whether an interactive terminal or a watching terminal) needs to play the interactive live broadcast, all the terminal equipment needs to have the capability of simultaneously processing decoding, mixing and encoding of multiple paths of audios and videos, namely, each terminal equipment has high configuration requirements, so that the equipment cost of the interactive live broadcast is greatly increased; however, in the method, in all terminal devices of each interactive live broadcast, only one terminal device, namely the main interactive terminal, has the capability of audio and video mixing processing and corresponding configuration, the auxiliary interactive terminal only needs to collect and upload audio and video data, and receive the mixed audio and video data generated by the main interactive terminal for decoding and playing, and the auxiliary interactive terminal and the watching terminal do not need to have the capability and configuration of audio and video mixing processing, so the method is beneficial to reducing the configuration requirement of the terminal device and reducing the equipment cost of the interactive live broadcast. 2.2, compared with the audio and video data forwarding flow in the prior art, the data forwarding flow of the method is simpler; originally, in the prior art, each terminal device needs to receive multiple paths of audio and video data for mixed processing, however, the method only needs a main interaction terminal to receive multiple paths of audio and video data for mixed processing, and only needs to receive one path of mixed audio and video data from the interaction terminal and a watching terminal for decoding and playing, so that the method is beneficial to the network bandwidth pressure of the terminal device; in the prior art, the server needs to forward the multi-channel audio and video data to each terminal device in the interactive live broadcast process, however, in the method of the invention, the server only needs to forward the multi-channel audio and video data to the main interactive terminal and only needs to transmit one channel of mixed audio and video data to other terminals or streaming media platforms, thereby being beneficial to reducing the network bandwidth pressure and processing load of the server, being beneficial to reducing the operation cost and ensuring the interactive live broadcast effect.
3) Regarding simplification of the data forwarding flow: the terminal equipment and the terminal equipment do not need to be connected independently, the distribution of data is completed by a server, and the communication complexity between the terminal equipment is reduced; specifically, the server allocates an annular buffer area for each terminal of each interactive live broadcast in a one-to-one correspondence manner, and can find corresponding audio and video data to forward by means of identification information of the terminal equipment.
4) The deployment of the streaming media service is simplified, a streaming media server built by the streaming media server or a third-party streaming media platform can be used, and only one path of mixed audio and video data of each interactive live broadcast needs to be received by the streaming media server or the streaming media platform aiming at each interactive live broadcast.
In summary, the multi-user audio/video interactive live broadcast data interaction method is beneficial to reducing the performance requirement of a server or the configuration requirement of terminal equipment, and is also beneficial to reducing the network bandwidth pressure, so that a good interactive live broadcast effect is ensured.
The embodiment of the invention also provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the method for interacting the live broadcast data of the multi-person audio/video interaction is realized when the processor executes the computer program.
The electronic device may operate only the step applied to the server, i.e. used as a server, or may operate only the step applied to the terminal device, i.e. used as a terminal device; of course, the steps applied to the server and the steps applied to the electronic device may be both performed. The electronic equipment is beneficial to reducing the performance requirement of a server or the configuration requirement of terminal equipment, and is also beneficial to reducing the network bandwidth pressure, so that a good interactive live broadcast effect is ensured.
The embodiment of the invention also provides a computer readable storage medium, and a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, the multi-person audio/video interactive live broadcast data interaction method of the embodiment of the invention is realized.
The computer readable storage medium helps to reduce the performance requirements of the server or the configuration requirements of the terminal equipment, and also helps to reduce the network bandwidth pressure, thereby ensuring good interactive live broadcast effect.
The above embodiments are only preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, but any insubstantial changes and substitutions made by those skilled in the art on the basis of the present invention are intended to be within the scope of the present invention as claimed.
Claims (10)
1. The method for interacting the live broadcast data of the multi-user audio/video interaction is characterized by comprising the following steps:
receiving a request instruction for initiating interactive live broadcast;
determining all terminal devices participating in the interactive live broadcast according to a request instruction, wherein one of the terminal devices is a master interactive terminal of the interactive live broadcast, and the other terminal devices are slave interactive terminals of the interactive live broadcast;
executing a data forwarding process for the interactive live broadcast, including:
receiving audio and video data acquired from an interactive terminal;
the audio and video data collected from the interaction terminal are forwarded to the main interaction terminal;
receiving mixed audio and video data generated by a main interactive terminal;
and forwarding the mixed audio and video data generated by the master interactive terminal to the slave interactive terminal.
2. The method for interacting live audio-visual data with multiple persons according to claim 1, wherein the data forwarding process further comprises:
and pushing the mixed audio and video data generated by the main interactive terminal to a viewing terminal or a streaming media platform.
3. The method for interacting multi-person audio-video interactive live data according to claim 1, further comprising:
allocating buffer areas for all terminal devices of the interactive live broadcast in a one-to-one correspondence manner;
the data forwarding flow further includes:
storing the audio and video data acquired from the interactive terminal into a buffer area corresponding to the interactive terminal;
when the audio and video data collected by the slave interaction terminal is forwarded to the master interaction terminal, the audio and video data are obtained from a buffer zone corresponding to the slave interaction terminal and forwarded to the master interaction terminal;
storing the mixed audio and video data generated by the main interactive terminal into a buffer area corresponding to the main interactive terminal;
when the mixed audio and video data generated by the master interactive terminal is forwarded to the slave interactive terminal, the mixed audio and video data is acquired from a buffer area corresponding to the master interactive terminal and forwarded to the slave interactive terminal.
4. The method for interactive live broadcast data interaction of multiple audio and video signals according to claim 3, wherein the buffer area is a ring buffer area.
5. The method for interacting the live broadcast data of the multi-user audio/video interaction is characterized by comprising the following steps:
when the method is used as a main interaction terminal in interaction live broadcast, the method comprises the following steps:
collecting audio and video data;
receiving audio and video data forwarded by a server and collected by an interaction terminal;
mixing the audio and video data acquired by the local machine with the audio and video data acquired from the interactive terminal to generate mixed audio and video data;
uploading the generated mixed audio and video data to a server, and decoding and playing;
when the method is used as a slave interaction terminal in the interaction live broadcast, the method comprises the following steps:
collecting audio and video data;
uploading audio and video data acquired by a local machine to a server;
the method comprises the steps of receiving mixed audio and video data forwarded by a server and generated by a main interaction terminal;
and decoding and playing the received mixed audio and video data.
6. The method for interacting with multi-person audio and video interactive live broadcast data according to claim 5, further comprising:
when the method is used as a main interaction terminal in the interaction live broadcast, the method further comprises the following steps:
pushing the generated mixed audio and video data to a streaming media platform.
7. The method for interacting with multi-person audio and video interactive live broadcast data according to claim 5, further comprising:
selecting all terminal devices participating in the interactive live broadcast, wherein one of the terminal devices is a master interactive terminal of the interactive live broadcast, and the other terminal devices are slave interactive terminals of the interactive live broadcast;
and sending a request instruction for initiating the interactive live broadcast to a server.
8. The method for interacting multi-user audio-video interactive live data according to claim 5, wherein the mixing process comprises:
separating each path of audio and video data into video data and audio data;
decoding each path of video data to obtain video decoding data, and decoding each path of audio data to obtain audio decoding data;
mixing each path of video decoding data to generate mixed video data, and mixing each path of audio decoding data to generate mixed audio data;
encoding the mixed video data to obtain mixed video encoded data, and encoding the mixed audio data to obtain mixed audio encoded data;
the mixed video encoded data and the mixed audio encoded data are combined to generate mixed audio-video data.
9. Electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the multi-person audio video interactive live data interaction method according to any of claims 1-8 when executing the computer program.
10. A computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements a multi-person audio video interactive live data interaction method as claimed in any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311727426.2A CN117714801B (en) | 2023-12-14 | 2023-12-14 | Multi-user audio/video interactive live broadcast data interaction method, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311727426.2A CN117714801B (en) | 2023-12-14 | 2023-12-14 | Multi-user audio/video interactive live broadcast data interaction method, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117714801A true CN117714801A (en) | 2024-03-15 |
| CN117714801B CN117714801B (en) | 2024-07-23 |
Family
ID=90143877
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311727426.2A Active CN117714801B (en) | 2023-12-14 | 2023-12-14 | Multi-user audio/video interactive live broadcast data interaction method, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117714801B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018050049A1 (en) * | 2016-09-18 | 2018-03-22 | 腾讯科技（深圳）有限公司 | Live broadcast method, system, server, and data storage medium |
| CN109561351A (en) * | 2018-12-03 | 2019-04-02 | 网易（杭州）网络有限公司 | Network direct broadcasting back method, device and storage medium |
| US20190364303A1 (en) * | 2018-05-22 | 2019-11-28 | Beijing Baidu Netcom Science Technology Co., Ltd. | Live broadcast processing method, apparatus, device, and storage medium |
| CN115695705A (en) * | 2021-07-26 | 2023-02-03 | Oppo广东移动通信有限公司 | Real-time communication method, computer-readable storage medium, and terminal device |
-
2023
- 2023-12-14 CN CN202311727426.2A patent/CN117714801B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018050049A1 (en) * | 2016-09-18 | 2018-03-22 | 腾讯科技（深圳）有限公司 | Live broadcast method, system, server, and data storage medium |
| US20190364303A1 (en) * | 2018-05-22 | 2019-11-28 | Beijing Baidu Netcom Science Technology Co., Ltd. | Live broadcast processing method, apparatus, device, and storage medium |
| CN109561351A (en) * | 2018-12-03 | 2019-04-02 | 网易（杭州）网络有限公司 | Network direct broadcasting back method, device and storage medium |
| CN115695705A (en) * | 2021-07-26 | 2023-02-03 | Oppo广东移动通信有限公司 | Real-time communication method, computer-readable storage medium, and terminal device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117714801B (en) | 2024-07-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4093860B2 (en) | Integrated delivery architecture for multipoint video conferencing and interactive broadcast systems | |
| JP5580302B2 (en) | Broadcast seeding for peer-to-peer networks | |
| CN102790923B (en) | Method, server of instant message and user terminal that user comment information is shared | |
| CN104756099B (en) | Add content and associated client device | |
| CN106507186B (en) | Media information switching method, server and storage medium | |
| CN111147362B (en) | Multi-user instant messaging method, system, device and electronic equipment | |
| JP2006521643A (en) | System and method for transmitting media-based files | |
| CN104093028B (en) | A kind of method and apparatus that capacity of equipment is negotiated | |
| US20110137438A1 (en) | Video conference system and method based on video surveillance system | |
| KR100937681B1 (en) | Communication module and process for inter-user communication, server comprising such a communication module, broadcasting set comprising such a server and storage medium stored thereon a computer program product executing such an inter-user communication process | |
| CN111131759B (en) | Real-time multimedia transmission system and use method thereof | |
| CN103618964A (en) | Method and system for providing television program through intelligent television | |
| CN113507574A (en) | System for processing and playing ultrahigh-definition video | |
| CN105120305A (en) | Method of playing picture in picture video on mobile terminal and system thereof | |
| CN106792244A (en) | Net cast method and device | |
| KR20170081517A (en) | Server and method for providing interactive broadcast | |
| US11431770B2 (en) | Method, system, apparatus, and electronic device for managing data streams in a multi-user instant messaging system | |
| CN117714801B (en) | Multi-user audio/video interactive live broadcast data interaction method, equipment and storage medium | |
| WO2019100631A1 (en) | Video playing method, apparatus and system, and storage medium | |
| CN112383794A (en) | Live broadcast method, live broadcast system, server and computer storage medium | |
| CN112532913A (en) | Video mixing method, video system and server | |
| CN111447458A (en) | Live broadcast system, method and device based on content explanation and live broadcast server | |
| CN113612728B (en) | Streaming media playing method, transmission equipment and system | |
| CN112218031A (en) | Video conference data transmission method and system | |
| US8862761B1 (en) | Method and system for forming an audio overlay for streaming content of a content distribution system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |