CN111050185B - Live broadcast room wheat-connected video mixing method, storage medium, electronic equipment and system - Google Patents

Live broadcast room wheat-connected video mixing method, storage medium, electronic equipment and system Download PDF

Info

Publication number
CN111050185B
CN111050185B CN201811199516.8A CN201811199516A CN111050185B CN 111050185 B CN111050185 B CN 111050185B CN 201811199516 A CN201811199516 A CN 201811199516A CN 111050185 B CN111050185 B CN 111050185B
Authority
CN
China
Prior art keywords
wheat
client
flow
microphone
initiating
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811199516.8A
Other languages
Chinese (zh)
Other versions
CN111050185A (en
Inventor
汪刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Douyu Network Technology Co Ltd
Original Assignee
Wuhan Douyu Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Douyu Network Technology Co Ltd filed Critical Wuhan Douyu Network Technology Co Ltd
Priority to CN201811199516.8A priority Critical patent/CN111050185B/en
Publication of CN111050185A publication Critical patent/CN111050185A/en
Application granted granted Critical
Publication of CN111050185B publication Critical patent/CN111050185B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/2187Live feed
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/233Processing of audio elementary streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/239Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests
    • H04N21/2393Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests involving handling client requests
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439Processing of audio elementary streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/485End-user interface for client configuration
    • H04N21/4858End-user interface for client configuration for modifying screen layout parameters, e.g. fonts, size of the windows

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

The invention discloses a live broadcast room microphone video flow mixing method, a storage medium, electronic equipment and a system, and relates to the field of application and development of internet mobile terminals. The flow mixing step comprises: the method comprises the steps that a wheat connection initiating client and a wheat connection client acquire wheat connection parameters from a server; the parameter verification is carried out on the parameter of the wheat connecting client by the wheat connecting initiating client and the parameter of the wheat connected client, and mixed flow data are sent to the server if the parameter of the wheat connecting is correct; and after receiving the mixed flow data of the microphone connecting initiating client and the microphone connecting receiving client, the server generates a composite data stream and a mixed flow video address and sends the composite data stream to the mixed flow video address. The plug flow step comprises: a spectator client obtains a stream pushing parameter; a spectator client generates a play window and requests a mixed-flow video address from a server; and after the mixed flow video address is obtained by the audience client, the composite data stream is obtained from the mixed flow video address, and the composite data stream is played in the microphone connecting window.

Description

Live broadcast room wheat-connected video mixing method, storage medium, electronic equipment and system
Technical Field
The invention relates to the field of application development of internet mobile terminals, in particular to a live broadcast room microphone video mixing method, a storage medium, electronic equipment and a system.
Background
With the rapid development of mobile terminals of mobile phones, the technology of connecting with the microphone is required to be used in many places in live broadcasting. So-called party connection is a technique of making a voice call or a video call between a main broadcasting and a main broadcasting/audience, and all users can see the video call or the voice call between the main broadcasting and the main broadcasting/audience, and the technique is called party connection technique. The technology that the main broadcast and the main broadcast/audience can make voice, all users can see the voice of the main broadcast is called voice microphone connection, the main broadcast and the main broadcast/audience carry out video, all users can see the video of the main broadcast and the main broadcast/audience, the technology is called video microphone connection, a client and a server are needed in the microphone connection process, the client and a player module need to cooperate in multiple aspects, if the three can not be well matched, the microphone connection process is slow to build, and the user experience is poor.
The technology which is the core of the wheat connecting technology is audio and video mixed flow, and the mixed flow process comprises four steps of decoding, mixed flow, encoding and flow pushing. The link of mixing flow comprises jitter buffering, aims to align the picture and synchronize the sound and the picture of a plurality of paths of flows, and simultaneously resists network jitter through buffering so as to achieve good effect after mixing into a path of flow. The mixed flow is an operation which relatively consumes computing resources, and is also a link which relatively consumes time. It is said to be computationally expensive because this process involves real-time transcoding of multiple streams, a relatively CPU-intensive matter. It is time consuming because this involves jitter buffering, which causes some delay. The time consumed by the real-time transcoding is generally lower than the time of one frame, and is relatively insignificant; but jitter buffering can take anywhere from 80 milliseconds to 1 second, depending on the network conditions. In live interactive live broadcasting, the delay caused by jitter buffering is critical to the experience of the audience.
However, before live mixed flow of live broadcasting, a user needs to wait for a long time from the time when the main broadcasting initiates the live broadcasting to the time when the main broadcasting establishes the live broadcasting, and the waiting is more influenced by the user experience compared with the jitter delay. Therefore, in order to solve the above problems, an optimized setup scheme for a connected video stream is needed, which is faster and more reliable while ensuring the setup of the connected video stream.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a live broadcast room microphone connecting video mixed flow method, a storage medium, electronic equipment and a system, which can more quickly and reliably ensure that the microphone connecting is established for audiences to watch and improve the watching experience of users.
In order to achieve the above object, in a first aspect, an embodiment of the present invention provides a live broadcast room microphone video mixing method, which includes an asynchronous mixing step and a plug flow step:
the step of mixing flow comprises:
the method comprises the following steps that a wheat connecting initiating client and a wheat connecting client acquire wheat connecting parameters from a server, wherein the wheat connecting parameters comprise: the ID of the current live broadcast room, the ID of a party initiating the wheat connection and the ID of a party being connected with the wheat;
the method comprises the steps that parameter verification is conducted on wheat connecting parameters by a wheat connecting initiating client and a wheat connected client, and mixed flow data are sent to a server if the wheat connecting parameters are not wrong;
the server receives mixed flow data of the microphone connecting initiating client and the microphone connecting receiving client, generates a composite data stream and a mixed flow video address, and sends the composite data stream to the mixed flow video address;
the plug flow step comprises:
the method comprises the following steps that a spectator client side obtains a push flow parameter, wherein the push flow parameter comprises: an ID of a live microphone connecting room, an ID of a microphone connecting initiator and a microphone connecting window parameter;
a spectator client generates a play window and requests a mixed-flow video address from a server;
and after the mixed flow video address is obtained by the audience client, the composite data stream is obtained from the mixed flow video address, and the composite data stream is played in the connecting window.
As an optional implementation, the method further comprises a wheat connection initiating step:
a connecting wheat initiating client initiates a connecting wheat request;
the server receives a wheat connecting request and is connected with wheat for matching;
and after the wheat connecting initiation step is finished, executing the mixed flow step, and before the mixed flow step is finished, executing the flow pushing step.
As an alternative embodiment, the step of mixing the flow and the step of pushing the flow are performed simultaneously after the step of initiating the wheat connection is completed.
As an optional embodiment, the parameter verification of the client on the parameters of the connecting wheat specifically includes the following steps:
checking whether the ID of the party initiating the connected wheat is the same as the ID of the party being connected, if so, reporting an error, and terminating the connected wheat;
and judging whether the wheat connecting parameter is a null value, if so, reporting an error and terminating wheat connecting.
As an optional implementation, the sending mixed flow data to the server specifically includes the following steps:
the method comprises the steps that a client acquires an ID of a live broadcast room which actively initiates connecting with the microphone, passively receives the ID of the live broadcast room which actively receives the connecting with the microphone, the type of the client of the live broadcast room which initiates the connecting with the microphone and the type of the live broadcast room which passively receives the connecting with the microphone;
the client side forms an array by the ID of the live broadcast room actively initiating the microphone connection, the ID of the live broadcast room passively receiving the microphone connection, the type of the client side of the live broadcast room initiating the microphone connection and the type of the live broadcast room passively receiving the microphone connection, and then sends the array to the server, and the server synthesizes received mixed flow data according to the data in the array.
As an optional embodiment, the client, the server and the audience client communicate through a socket, and play the microphone-connected video to the user is realized on php.
As an optional embodiment, the microphone connecting window parameter includes: window coordinates, window height, window width.
In a second aspect, an embodiment of the present invention provides a mixed flow system for live broadcast room continuous wheat video, which includes a mixed flow module and a plug flow module that are independent of each other:
the mixed flow module is used for:
obtaining a Lian Mai parameter from a server, wherein the Lian Mai parameter comprises: the ID of the current live broadcast room, the ID of a party initiating the wheat connection and the ID of a party being connected with the wheat;
parameter verification is carried out on the wheat connecting parameters, and if the wheat connecting parameters are not wrong, mixed flow data are sent to a server;
after mixed flow data of a microphone connection initiating client and a microphone connection receiving client are received, generating a composite data stream and a mixed flow video address, and sending the composite data stream to the mixed flow video address;
the plug flow module is used for:
acquiring plug flow parameters, wherein the plug flow parameters comprise: an ID of a live microphone connecting room, an ID of a microphone connecting initiator and a microphone connecting window parameter;
generating a playing window and requesting a mixed flow video address from a server;
and after the mixed flow video address is obtained, obtaining the composite data stream from the mixed flow video address, and playing the composite data stream in the microphone connecting window.
Based on the second aspect, as an optional implementation manner, the system further includes:
the system comprises a connecting microphone initiating module, a connecting microphone initiating module and a connecting microphone initiating module, wherein the connecting microphone initiating module is used for a connecting microphone initiating client terminal to initiate a connecting microphone request;
the matching module is used for receiving the connecting request and carrying out connecting matching;
and the execution control module is used for controlling the mixed flow module and the plug flow module to start to execute the plug flow module before the mixed flow module is executed.
Based on the second aspect, as a preferred embodiment, the flow mixing module and the flow pushing module are executed simultaneously. In a third aspect, an embodiment of the present invention further provides a storage medium, where a computer program is stored on the storage medium, and when being executed by a processor, the computer program implements the method in the embodiment of the first aspect.
In a fourth aspect, an embodiment of the present invention further provides an electronic device, which includes a memory and a processor, where the memory stores a computer program running on the processor, and the processor executes the computer program to implement the method in the first aspect.
Compared with the prior art, the invention has the advantages that:
(1) the mixed flow method, the storage medium, the electronic equipment and the system for the live broadcast room live broadcast video, provided by the invention, have the advantages that the mixed flow step and the push flow step which should be sequentially carried out are independent and are asynchronously carried out, so that the live broadcast video can be more quickly seen or the live broadcast voice can be more quickly heard from the main broadcast to the audience after the live broadcast starts to connect the broadcast. The mixed flow video address is generated in the mixed flow step, the composite data flow generated in the mixed flow step is obtained from the mixed flow video address in the plug flow step, the two steps are mutually independent and can be carried out asynchronously, namely, when the two steps are operated independently, the two steps cannot be influenced mutually, and after the operation is finished, the two steps can be mutually associated to ensure the stable and rapid operation of the wheat.
(2) The live broadcast room wheat connection video mixed flow method, the storage medium, the electronic equipment and the system execute the mixed flow step after the wheat connection step is initiated, and execute the flow pushing step before the mixed flow step is completed, so that the two steps are at least partially overlapped, and the steps which should be sequentially performed are ensured. Preferably, the flow mixing step and the flow pushing step are performed simultaneously, so that the maximized overlapping can be ensured, the waiting time of a user is shortened to the maximum degree, and the user experience is improved
(3) The live broadcast room microphone video mixed flow method, the storage medium, the electronic equipment and the system communicate through the socket, and the microphone video playing to the user can be faster and more reliable compared with other microphone connecting modes in php.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings corresponding to the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a live broadcast room wheat-connected video mixing method according to the present invention;
fig. 2 is a schematic structural diagram of a direct broadcasting room wheat-connected video mixed flow system of the present invention.
In the figure: 1-mixed flow module, 2-plug flow module.
Detailed Description
Embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.
The embodiment of the invention provides a live broadcast room wheat-connected video mixed flow method, a storage medium, electronic equipment and a system.
In order to achieve the technical effect, the general idea of the application is as follows:
asynchronous flow mixing and flow pushing steps:
the step of mixing flow comprises:
the method comprises the following steps that a wheat connecting initiating client and a wheat connecting client acquire wheat connecting parameters from a server, wherein the wheat connecting parameters comprise: the ID of the current live broadcast room, the ID of a party initiating the wheat connection and the ID of a party being connected with the wheat;
the method comprises the steps that parameter verification is conducted on wheat connecting parameters by a wheat connecting initiating client and a wheat connected client, and mixed flow data are sent to a server if the wheat connecting parameters are not wrong;
the server receives mixed flow data of the microphone connecting initiating client and the microphone connecting receiving client, generates a composite data stream and a mixed flow video address, and sends the composite data stream to the mixed flow video address;
the plug flow step comprises:
the method comprises the following steps that a spectator client side obtains a push flow parameter, wherein the push flow parameter comprises: an ID of a live microphone connecting room, an ID of a microphone connecting initiator and a microphone connecting window parameter;
a spectator client generates a play window and requests a mixed-flow video address from a server;
and after the mixed flow video address is obtained by the audience client, the composite data stream is obtained from the mixed flow video address, and the composite data stream is played in the microphone connecting window.
In conclusion, the invention can greatly shorten the time for the audience to wait for the continuous microphone video. In the prior art, the live broadcast of continuous wheat needs to be gradually completed: the background server receives the data streams, mixes the data streams to obtain a synthesized data stream, then informs the audience to start initialization to prepare for playing the video, and sends the synthesized data stream to the audience after the audience finishes the preparation.
The invention makes the mixed flow step and the pushed flow step independent, ensures that the pushed flow step and the mixed flow step are not mutually influenced before the completion, and can realize direct wheat seeding after the completion of the two steps. Just because the two steps are performed independently of each other, i.e. asynchronously, the two steps are performed in an overlapping manner, and the overlapping portion is a relatively short time.
For the independence of the two steps, the method adopts a mode that after the mixed flow step obtains the required parameters, a mixed flow video address is generated, and a composite data stream is sent to the mixed flow video address; in the stream pushing step, after the audience client finishes the generation of the playing window, the mixed-flow video address is obtained from the server, and then the stream pulling can be carried out.
In order to better understand the technical solution, the following detailed description is made with reference to specific embodiments.
Example one
As shown in fig. 1, an embodiment of the present invention provides a live broadcast room continuous wheat video mixing method, which includes:
asynchronous flow mixing and flow pushing steps:
the step of mixing flow comprises:
s10: the method comprises the following steps that a wheat connecting initiating client and a wheat connecting client acquire wheat connecting parameters from a server, wherein the wheat connecting parameters comprise: the method comprises the steps of obtaining a current live broadcasting room ID, a connecting party ID and a connected party ID;
s11: the method comprises the steps that parameter verification is conducted on wheat connecting parameters by a wheat connecting initiating client and a wheat connected client, and mixed flow data are sent to a server if the wheat connecting parameters are not wrong;
s12: the server receives mixed flow data of the microphone connecting initiating client and the microphone connecting receiving client, generates a composite data stream and a mixed flow video address, and sends the composite data stream to the mixed flow video address;
in the live broadcast of the continuous wheat, firstly the continuous wheat between the anchor broadcasters is needed, but the invention makes the continuous wheat between the anchor broadcasters independent, thereby ensuring that the subsequent stream pushing step is not influenced, and ensuring that the two steps are relatively independent and can be asynchronously carried out.
The mixed-channel video address is generated instead of generating a loading notice after mixed flow is finished to establish a playing window for a viewer client, so that the two steps can be asynchronously carried out. After the server receives the video and audio used by the microphone connecting initiating client and the microphone connected client, the server generates a mixed flow video address and sends a composite data stream through the mixed flow video address.
Firstly, the connecting wheat between the main broadcasts comprises a connecting wheat initiating client and a connected wheat client, and the connecting wheat can be carried out after the master-slave sequence of the connecting wheat initiating client and the connected wheat client is confirmed. Therefore, data transmission and data verification are required in the flow mixing step. And the whole data of the continuous microphone is sent to the server, the server checks one data, and after the data is checked to be qualified, the client is informed to send the video and audio for continuous microphone, so that the complexity is too high. The invention directly sends the data to be verified to the connected-microphone initiating client and the connected-microphone client, the two clients directly verify the data, and directly send the video and the audio with mixed flow requirements after the verification is finished, so that the method is more rapid.
It should be noted that, here, the connecting initiating client and the connected client are clients that send video streams, and may be clients that are used by two anchor broadcasters, that is, connecting between the two anchor broadcasters, and transmitting video and/or audio streams to a server for mixing. Or the client used by the main broadcast and the audience, namely the main broadcast and the audience/fan thereof carry out microphone connection and transmit video and/or audio streams to the server for mixed flow. Meanwhile, the number of the connected clients can be multiple, and in order to achieve the fastest effect of connecting the wheat, the method preferably performs one-to-one wheat connection.
As a preferred embodiment, the parameter verification of the client on the parameters of the linkman specifically includes the following steps:
s111, checking whether the ID of the party initiating the connecting wheat is the same as the ID of the party being connected, if so, reporting an error, and terminating the connecting wheat.
S112, judging whether the wheat connecting parameter is a null value, if so, reporting an error and terminating wheat connecting.
When checking the wheat connection data, firstly, it is ensured that the main broadcast and the main broadcast do not connect the wheat. Meanwhile, since the continuous microphone has more requirements on each parameter, necessary parameters are ensured not to be defective, and therefore null value verification is carried out on data.
As another preferred embodiment, the sending of the mixed flow data to the server specifically comprises the steps of:
the method comprises the steps that a client acquires an ID of a live broadcast room which actively initiates connecting with the microphone, passively receives the ID of the live broadcast room which actively receives the connecting with the microphone, the type of the client of the live broadcast room which initiates the connecting with the microphone and the type of the live broadcast room which passively receives the connecting with the microphone;
the client side forms an array by the ID of the live broadcast room actively initiating the microphone connection, the ID of the live broadcast room passively receiving the microphone connection, the type of the client side of the live broadcast room initiating the microphone connection and the type of the live broadcast room passively receiving the microphone connection, and then sends the array to the server, and the server synthesizes received mixed flow data according to the data in the array.
The data required by mixed flow is sent to the server one by one, which may be partially lost, and the sending and confirmation one by one also causes the synthesis of the continuous-microphone video to be slow. And the parameters required by the mixed flow are combined to obtain an array, and the array is sent to a server through the array at one time, so that the interaction between the client and the server can be simplified, and more time is saved.
The plug flow step comprises:
s20: the method comprises the following steps that a spectator client side obtains a push flow parameter, wherein the push flow parameter comprises: an ID of a live microphone connecting room, an ID of a microphone connecting initiator and a microphone connecting window parameter;
s21: a spectator client generates a play window and requests a mixed-flow video address from a server;
s22: and after the mixed flow video address is obtained by the audience client, the composite data stream is obtained from the mixed flow video address, and the composite data stream is played in the microphone connecting window.
The plug flow step and the mixed flow step are carried out asynchronously, so that the waiting time of audiences in a live broadcast room can be shortened. The flow pushing step is independent relative to the flow mixing step, can directly prepare by itself, and automatically pulls the synthetic data flow generated by the server after the mixed flow video address is obtained.
Specifically, the window parameters of the microphone comprise: window coordinates, window height, window width. By the coordinates of the window, the spectator client can know the actual position of the playing window, and then the playing window is established by the height and the width of the window.
It should be noted that, here, the audience client includes a client performing a live connection and a client not performing a live connection. The client end for connecting the wheat needs to pull the synthetic data stream from the mixed-flow video address to enable wheat connecting personnel to check whether the wheat connecting is normally carried out, and for the client end for connecting the wheat which watches the wheat connecting in the live broadcasting room and does not carry out the wheat connecting, the client end pulls the synthetic data stream in the mixed-flow video base to enable the client end to normally watch the fans and other audiences in the live broadcasting room.
As a preferred embodiment, the method further comprises the following steps of:
a connecting wheat initiating client initiates a connecting wheat request;
the server receives a wheat connecting request and is connected with wheat for matching;
and after the wheat connecting initiation step is finished, executing the mixed flow step, and before the mixed flow step is finished, executing the flow pushing step.
Before the flow mixing step is completed, the flow pushing step is executed to ensure that the two steps are mutually overlapped, and the time of waiting for live video and audio of the connected wheat is reduced for a user. Although the two steps are performed asynchronously, the next step can be performed after the next step is completed, so that preferably, after the wheat connection initiating step is completed, the step is performed, and before the mixed flow step is completed, the flow pushing step is performed, so that the two steps are actively at least partially parallel, and the time is saved more relatively one by one.
Further, the mixed flow step and the pushing flow step are executed simultaneously after the wheat connecting initiating step is finished.
And the flow mixing step and the flow pushing step are executed simultaneously, so that the two steps can be maximally parallel and overlapped, and the maximum time can be saved. Regardless of the long time consumption of the mixed flow step or the push flow step, as long as the two steps are started simultaneously, the two steps are already started to be parallel when starting, and are parallel after the other step which consumes shorter time is completed, through the relevant steps, the parallel of the two steps with the longest time length can be ensured, namely the execution time of the step which consumes the shortest time is the parallel time of the steps, and the total execution time of the two steps is the execution time of the step which consumes the longest time. After the maximum parallelism, the time for the parallelism is saved, and the execution time of the step which takes the shortest time is saved. The experience of the user is improved to the maximum extent.
As a preferred embodiment, the client, the server and the viewer client communicate with each other through a Socket, and play a microphone video Socket, also commonly referred to as a "Socket", describing an IP address and a port, to the user is implemented on php, and is a handle of a communication chain, which can be used to implement communication between different virtual machines or different computers. The method can ensure the completeness and rapidness of communication between the client and the server, and the foreign language name of PHP is PHP, Hypertext Preprocessor, Chinese name: the grammar of the hypertext preprocessor absorbs the characteristics of C language, Java and Perl, is beneficial to learning, has wide use and is mainly suitable for the field of Web development. The PHP unique syntax mixes the C, Java, Perl, and PHP self-created syntax. It can execute dynamic web pages faster than CGI or Perl. Therefore, the pull stream can be established more quickly and reliably based on the fast speed of the socket and the php.
Based on the same inventive concept, the present application provides the second embodiment, which is as follows.
Example two
As shown in fig. 2, an embodiment of the present invention provides a mixed flow system for live broadcast room and live broadcast room, including a mixed flow module 1 and a plug flow module 2 that are independent of each other:
the mixed flow module 1 is used for:
obtaining a Lian Mai parameter from a server, wherein the Lian Mai parameter comprises: the method comprises the steps of obtaining a current live broadcasting room ID, a connecting party ID and a connected party ID;
parameter verification is carried out on the wheat connecting parameters, and if the wheat connecting parameters are not wrong, mixed flow data are sent to a server;
after mixed flow data of a microphone connection initiating client and a microphone connection receiving client are received, generating a composite data stream and a mixed flow video address, and sending the composite data stream to the mixed flow video address;
the plug flow module 2 is used for:
acquiring plug flow parameters, wherein the plug flow parameters comprise: an ID of a live microphone connecting room, an ID of a microphone connecting initiator and a microphone connecting window parameter;
generating a playing window and requesting a mixed flow video address from a server;
and after the mixed flow video address is obtained, obtaining the composite data stream from the mixed flow video address, and playing the composite data stream in the microphone connecting window.
As an optional implementation, the system further comprises:
the system comprises a connecting microphone initiating module, a connecting microphone initiating client and a connecting microphone sending module, wherein the connecting microphone initiating module is used for initiating a connecting microphone request by a connecting microphone initiating client;
the matching module is used for receiving the connecting request and carrying out connecting matching;
and the execution control module is used for controlling the mixed flow module and the plug flow module to start to execute the plug flow module before the mixed flow module is executed.
As a preferred embodiment, the flow mixing module and the flow pushing module are performed simultaneously. Various modifications and specific examples of the foregoing embodiments of the method are also applicable to the system of the present embodiment, and the detailed description of the foregoing method will make apparent to those skilled in the art the implementation of the system of the present embodiment and the corresponding benefits of these methods, so that they will not be described in detail herein for the sake of brevity of this description.
Based on the same inventive concept, the present application provides the third embodiment.
EXAMPLE III
A third embodiment of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a video recommendation method as provided in any of the embodiments of the present invention, the method comprising asynchronous streaming and streaming steps:
the step of mixing flow comprises:
the method comprises the following steps that a wheat connecting initiating client and a wheat connecting client acquire wheat connecting parameters from a server, wherein the wheat connecting parameters comprise: the ID of the current live broadcast room, the ID of a party initiating the wheat connection and the ID of a party being connected with the wheat;
the method comprises the steps that parameter verification is conducted on wheat connecting parameters by a wheat connecting initiating client and a wheat connected client, and mixed flow data are sent to a server if the wheat connecting parameters are not wrong;
the server receives mixed flow data of the microphone connecting initiating client and the microphone connecting receiving client, generates a composite data stream and a mixed flow video address, and sends the composite data stream to the mixed flow video address;
the plug flow step comprises:
the method comprises the following steps that a spectator client side obtains a push flow parameter, wherein the push flow parameter comprises: an ID of a live microphone connecting room, an ID of a microphone connecting initiator and a microphone connecting window parameter;
a spectator client generates a play window and requests a mixed-flow video address from a server;
and after the mixed flow video address is obtained by the audience client, the composite data stream is obtained from the mixed flow video address, and the composite data stream is played in the microphone connecting window.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer-readable storage medium may be, for example but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the particular embodiments described herein, and that various obvious changes, rearrangements and substitutions will now be apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Based on the same inventive concept, the present application provides the fourth embodiment.
Example four
The fourth embodiment of the present invention further provides an electronic device, which includes a memory and a processor, where the memory stores a computer program running on the processor, and the processor executes the computer program to implement all or part of the method steps in the first embodiment.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general purpose processor may be a microprocessor or the processor may be any conventional processor or the like which is the control center for the computer device and which connects the various parts of the overall computer device using various interfaces and lines.
The memory may be used to store the computer programs and/or modules, and the processor may implement various functions of the computer device by running or executing the computer programs and/or modules stored in the memory and invoking data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, video data, etc.) created according to the use of the cellular phone, etc. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
Generally speaking, the live broadcast room wheat-connected video mixed flow method, the storage medium, the electronic device and the system provided by the embodiment of the invention divide the wheat-connected live broadcast step which should be gradually carried out to obtain the mixed flow step and the push flow step which can be carried out asynchronously, can shorten the time required by a viewer for waiting for a wheat-connected video after waiting for a main broadcast for wheat connection, and improve the user experience.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (8)

1. A live broadcast room continuous wheat video flow mixing method is characterized by comprising an asynchronous flow mixing step and a flow pushing step:
the step of mixing flow comprises:
the method comprises the following steps that a wheat connecting initiating client and a wheat connecting client obtain wheat connecting parameters from a server, wherein the wheat connecting parameters comprise: the ID of the current live broadcast room, the ID of a party initiating the wheat connection and the ID of a party being connected with the wheat;
the method comprises the steps that a wheat connecting initiating client and a wheat connected client carry out parameter verification on wheat connecting parameters, and if the wheat connecting parameters are not wrong, the wheat connecting initiating client and the wheat connected client send mixed flow data of the wheat connecting initiating client and the wheat connected client to a server;
the server receives mixed flow data of the microphone connecting initiating client and the microphone connecting receiving client, generates a composite data stream and a mixed flow video address, and sends the composite data stream to the mixed flow video address;
the flow mixing step and the flow pushing step are carried out asynchronously, and the flow pushing step comprises the following steps:
the method comprises the following steps that a spectator client side obtains a push flow parameter, wherein the push flow parameter comprises: an ID of a live microphone connecting room, an ID of a microphone connecting initiator and a microphone connecting window parameter;
a spectator client generates a play window and requests a mixed-flow video address from a server;
after the mixed-flow video address is obtained by the audience client, the composite data stream is obtained from the mixed-flow video address, and the composite data stream is played in the microphone connecting window;
further comprises a wheat connection initiating step:
the connecting client initiates a connecting request;
the server receives a wheat connecting request and is connected with wheat for matching;
after the wheat connecting initiation step is finished, the mixed flow step is executed, and before the mixed flow step is finished, the flow pushing step is executed;
and after the wheat connecting initiating step is completed, the mixed flow step and the flow pushing step are simultaneously started.
2. The method of claim 1, wherein the client performing parameter verification on the parameters of the connecting wheat specifically comprises the following steps:
checking whether the ID of the party initiating the connected wheat is the same as the ID of the party being connected, if so, reporting an error, and terminating the connected wheat;
and judging whether the wheat connecting parameter has a null value, if so, reporting an error and terminating wheat connecting.
3. The method of claim 1, wherein the sending the mixed flow data to the server specifically comprises the steps of:
the method comprises the steps that a client side obtains an ID of a live broadcast room for actively initiating a microphone connection, an ID of a live broadcast room for passively receiving the microphone connection, the type of the client side of the live broadcast room for initiating the microphone connection and the type of the live broadcast room for passively receiving the microphone connection;
the client side forms an array by the ID of the live broadcast room actively initiating the microphone connection, the ID of the live broadcast room passively receiving the microphone connection, the type of the client side of the live broadcast room initiating the microphone connection and the type of the live broadcast room passively receiving the microphone connection, and then sends the array to the server, and the server synthesizes received mixed flow data according to the data in the array.
4. The method of claim 1, wherein:
the client, the server and the audience client communicate through a socket, and the playing of the microphone connecting video to the user is realized on php.
5. The method of claim 1, wherein the Lianmai window parameters comprise: window coordinates, window height, window width.
6. A storage medium having a computer program stored thereon, characterized in that: the computer program, when executed by a processor, implements the method of any of claims 1 to 5.
7. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program that runs on the processor, characterized in that: the processor, when executing the computer program, implements the method of any of claims 1 to 5.
8. The utility model provides a live room even wheat video mixed flow system which characterized in that, it includes mutually independent mixed flow module, pushes away a class module and even wheat and initiates the module:
the mixed flow module is used for:
the method comprises the following steps that a wheat connecting initiating client and a wheat connecting client acquire wheat connecting parameters from a server, wherein the wheat connecting parameters comprise: the ID of the current live broadcast room, the ID of a party initiating the wheat connection and the ID of a party being connected with the wheat;
the method comprises the steps that parameter verification is conducted on wheat connecting parameters by a wheat connecting initiating client and a wheat connected client, and mixed flow data are sent to a server if the wheat connecting parameters are not wrong;
after receiving mixed flow data of a microphone connecting initiating client and a microphone connecting receiving client, a server generates a composite data stream and a mixed flow video address and sends the composite data stream to the mixed flow video address;
the plug flow module is used for:
acquiring plug flow parameters, wherein the plug flow parameters comprise: an ID of a live microphone connecting room, an ID of a microphone connecting initiator and a microphone connecting window parameter;
a spectator client generates a play window and requests a mixed-flow video address from a server;
after the mixed-flow video address is obtained by the audience client, the composite data stream is obtained from the mixed-flow video address, and the composite data stream is played in the microphone connecting window;
the microphone connecting initiating module is used for:
initiating a wheat connecting request;
the server receives a wheat connecting request and is connected with wheat for matching;
after the wheat connection initiating step is completed, a mixed flow step is executed, and before the mixed flow step is completed, a flow pushing step is executed;
and after the wheat connecting initiating step is completed, the mixed flow step and the flow pushing step are simultaneously started.
CN201811199516.8A 2018-10-15 2018-10-15 Live broadcast room wheat-connected video mixing method, storage medium, electronic equipment and system Active CN111050185B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811199516.8A CN111050185B (en) 2018-10-15 2018-10-15 Live broadcast room wheat-connected video mixing method, storage medium, electronic equipment and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811199516.8A CN111050185B (en) 2018-10-15 2018-10-15 Live broadcast room wheat-connected video mixing method, storage medium, electronic equipment and system

Publications (2)

Publication Number Publication Date
CN111050185A CN111050185A (en) 2020-04-21
CN111050185B true CN111050185B (en) 2022-08-16

Family

ID=70230575

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811199516.8A Active CN111050185B (en) 2018-10-15 2018-10-15 Live broadcast room wheat-connected video mixing method, storage medium, electronic equipment and system

Country Status (1)

Country Link
CN (1) CN111050185B (en)

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111526386A (en) * 2020-05-06 2020-08-11 北京三体云时代科技有限公司 Data transmission method and device based on auxiliary mixed screen equipment and data transmission system
CN113840170B (en) * 2020-06-23 2023-06-16 武汉斗鱼网络科技有限公司 Method and device for live broadcast of wheat
CN111836074B (en) * 2020-07-23 2022-09-09 北京达佳互联信息技术有限公司 Live wheat-connecting method and device, electronic equipment and storage medium
CN112040259A (en) * 2020-08-28 2020-12-04 广州华多网络科技有限公司 Method, server, system, storage medium and equipment for continuous playing
CN112135155B (en) * 2020-09-11 2022-07-19 上海七牛信息技术有限公司 Audio and video connecting and converging method and device, electronic equipment and storage medium
CN112291583B (en) * 2020-10-29 2022-12-27 北京达佳互联信息技术有限公司 Live broadcast and microphone connecting method and device, server, terminal and storage medium
CN112738610A (en) * 2020-12-25 2021-04-30 北京达佳互联信息技术有限公司 Display control method and device of multimedia data, electronic equipment and storage medium
CN112738638A (en) * 2020-12-25 2021-04-30 北京百度网讯科技有限公司 Live wheat-connecting method, device, equipment, storage medium and product
CN113473162B (en) * 2021-04-06 2023-11-03 北京沃东天骏信息技术有限公司 Media stream playing method, device, equipment and computer storage medium
CN115484469B (en) * 2021-06-15 2024-01-09 北京字节跳动网络技术有限公司 Wheat connecting system, method, device, equipment and storage medium
CN113810775B (en) * 2021-09-13 2023-04-07 上海哔哩哔哩科技有限公司 Voice and video connection system and method
CN114124911B (en) * 2021-11-25 2024-05-10 腾讯音乐娱乐科技(深圳)有限公司 Live echo cancellation method, computer readable storage medium and electronic device
CN116962803A (en) * 2022-04-14 2023-10-27 北京字跳网络技术有限公司 Method, device, equipment and medium for processing direct broadcast audio and video stream

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103841353A (en) * 2014-02-24 2014-06-04 广州华多网络科技有限公司 Video interaction method, terminal, server and system
CN106254899A (en) * 2016-08-16 2016-12-21 网宿科技股份有限公司 The control method of a kind of live even wheat and system
CN106658205A (en) * 2016-11-22 2017-05-10 广州华多网络科技有限公司 Studio video streaming synthesis control method, device and terminal equipment
CN107027048A (en) * 2017-05-17 2017-08-08 广州市千钧网络科技有限公司 A kind of live even wheat and the method and device of information displaying

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7542814B2 (en) * 2004-06-01 2009-06-02 Craig Barr Home entertainment apparatus and method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103841353A (en) * 2014-02-24 2014-06-04 广州华多网络科技有限公司 Video interaction method, terminal, server and system
CN106254899A (en) * 2016-08-16 2016-12-21 网宿科技股份有限公司 The control method of a kind of live even wheat and system
CN106658205A (en) * 2016-11-22 2017-05-10 广州华多网络科技有限公司 Studio video streaming synthesis control method, device and terminal equipment
CN107027048A (en) * 2017-05-17 2017-08-08 广州市千钧网络科技有限公司 A kind of live even wheat and the method and device of information displaying

Also Published As

Publication number Publication date
CN111050185A (en) 2020-04-21

Similar Documents

Publication Publication Date Title
CN111050185B (en) Live broadcast room wheat-connected video mixing method, storage medium, electronic equipment and system
CN112714330B (en) Gift presenting method and device based on live broadcast with wheat and electronic equipment
CN111836074B (en) Live wheat-connecting method and device, electronic equipment and storage medium
CN111107441B (en) Microphone connection communication establishing method, storage medium, electronic device and system
CN110267064B (en) Audio playing state processing method, device, equipment and storage medium
CN110910860B (en) Online KTV implementation method and device, electronic equipment and storage medium
CN111147362B (en) Multi-user instant messaging method, system, device and electronic equipment
CN111028818B (en) Chorus method, apparatus, electronic device and storage medium
CN111163330A (en) Live video rendering method, device, system, equipment and storage medium
CN112565802B (en) Live interaction method, system, server and storage medium
CN113225577A (en) Live stream processing method, device and system, electronic equipment and storage medium
CN113365094A (en) Live broadcast-based push stream data processing method, computing device and storage medium
CN114553839A (en) RTC data processing method and device
CN113037751B (en) Method and system for creating audio/video receiving stream
CN112383794B (en) Live broadcast method, live broadcast system, server and computer storage medium
CN113691516A (en) Streaming media data transmission method and device, electronic equipment and storage medium
WO2017071642A1 (en) Media playback method, device and computer storage medium
CN112786070A (en) Audio data processing method and device, storage medium and electronic equipment
CN112201264A (en) Audio processing method and device, electronic equipment, server and storage medium
CN112055227A (en) Cloud game interaction method, system, device, storage medium and electronic equipment
TWI697236B (en) Video conference audio and video sharing method
CN113411636A (en) Live wheat-connecting method and device, electronic equipment and computer-readable storage medium
CN113747181A (en) Network live broadcast method, live broadcast system and electronic equipment based on remote desktop
CN112532719A (en) Information flow pushing method, device, equipment and computer readable storage medium
CN105744344A (en) Play control method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant