CN115767128A

CN115767128A - Multimedia stream switching method, device and system

Info

Publication number: CN115767128A
Application number: CN202211558066.3A
Authority: CN
Inventors: 张捷
Original assignee: Shanghai Bilibili Technology Co Ltd
Current assignee: Shanghai Bilibili Technology Co Ltd
Priority date: 2022-12-06
Filing date: 2022-12-06
Publication date: 2023-03-07

Abstract

The application provides a multimedia stream switching method, a device and a system, wherein the multimedia stream switching method comprises the following steps: receiving a first multimedia stream sent by a first user terminal, caching the first multimedia stream based on set delay time, and pushing the cached first multimedia stream based on the current moment; in the process of pushing the first multimedia stream, if an interaction request of a first user end and a second user end is received, acquiring a second multimedia stream of the second user end; performing mixed flow treatment on the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, wherein the mixed flow time for performing mixed flow treatment is less than or equal to the set delay time; and switching the first multimedia stream into the mixed multimedia stream for pushing. The problem of cutoff and resource waste are avoided, the multimedia contents corresponding to the output mixed multimedia stream can be completely aligned and smooth without pause phenomenon, and the smoothness of multimedia switching is improved.

Description

Multimedia stream switching method, device and system

Technical Field

The present application relates to the field of computer technologies, and in particular, to a multimedia stream switching method. The application also relates to a multimedia stream switching apparatus, a multimedia stream switching system, a computing device, and a computer-readable storage medium.

Background

With the rapid development of computer technology, the live broadcast technology is more mature. In order to enhance the interactivity of live broadcast, live broadcast and live broadcast are carried out simultaneously.

And starting to connect the live broadcasting in the live broadcasting process of the anchor, wherein the server needs to mix the live broadcasting streams of the anchor and the live broadcasting streams of the users participating in the live broadcasting, and then, the mixed live broadcasting streams are pushed to the audience client for watching the live broadcasting of each live broadcasting client. Therefore, the live content of the user microphone connection interaction of each microphone connection client can be displayed at the audience client.

However, since the server needs to mix the live streams of the anchor and the microphone connecting users, it takes a certain time to directly switch the multimedia stream corresponding to the anchor to the live stream after the mixing processing of the server, which may cause the stream to be interrupted or the picture to be stuck, thereby reducing the experience during microphone connecting. Therefore, there is a need to provide a smoother scheme for multimedia stream switching.

Disclosure of Invention

In view of this, an embodiment of the present application provides a multimedia stream switching method. The application also relates to a multimedia stream switching device, a multimedia stream switching system, a computing device and a computer readable storage medium, so as to solve the technical defect that the multimedia stream switching is stuck in the prior art.

According to a first aspect of an embodiment of the present application, a multimedia stream switching method is provided, including:

receiving a first multimedia stream sent by a first user terminal, caching the first multimedia stream based on a set delay time, and pushing the cached first multimedia stream based on the current moment;

in the pushing process of the first multimedia stream, if an interaction request of a first user side and a second user side is received, a second multimedia stream of the second user side is obtained;

performing mixed flow treatment on the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, wherein the mixed flow time for performing mixed flow treatment is less than or equal to the set delay time;

and switching the first multimedia stream into the mixed multimedia stream for pushing.

Optionally, the pushing the cached first multimedia stream based on the current time includes:

subtracting the set delay time from the current time to obtain the time to be pushed;

and pushing the first multimedia stream corresponding to the moment to be pushed.

Optionally, the first multimedia stream is composed of a plurality of first multimedia frames carrying timestamps, and the second multimedia stream is composed of a plurality of second multimedia frames carrying timestamps;

the mixing the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, including:

and mixing the first multimedia frame and the second multimedia frame which carry the same timestamp to obtain a mixed multimedia frame carrying the timestamp, wherein a plurality of mixed multimedia frames form a mixed multimedia stream.

Optionally, the first multimedia stream is composed of a plurality of first multimedia frames carrying timestamps, and the mixed multimedia stream is composed of a plurality of mixed multimedia frames carrying timestamps;

the switching the first multimedia stream into the mixed multimedia stream for pushing comprises:

comparing the cached first multimedia frame to be pushed with the obtained mixed multimedia frame, and checking whether a target mixed multimedia frame with the same timestamp as that carried by the first multimedia frame to be pushed exists;

and if so, switching the cached first multimedia frame to be pushed into the target mixed multimedia frame for pushing.

Optionally, the first multimedia frame carries identification information, where the identification information indicates whether the first multimedia frame is a key frame;

the switching the cached and to-be-pushed first multimedia frame to the target mixed multimedia frame for pushing comprises:

identifying whether the cached first multimedia frame to be pushed is a key frame or not according to identification information carried by the cached first multimedia frame to be pushed;

Optionally, the mixing the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream includes:

decoding the first multimedia stream and the second multimedia stream respectively to obtain a decoded first multimedia stream and a decoded second multimedia stream;

mixing the decoded first multimedia stream and the decoded second multimedia stream to obtain an initial mixed multimedia stream;

and coding the initial mixed multimedia stream to obtain a target mixed multimedia stream.

Optionally, after the switching the first multimedia stream into the hybrid multimedia stream for pushing, the method further includes:

and if the interaction disconnection between the first user terminal and the second user terminal is detected, switching the mixed multimedia stream into the first multimedia stream for pushing.

According to a second aspect of the embodiments of the present application, there is provided a multimedia stream switching apparatus, including:

the receiving module is configured to receive a first multimedia stream sent by a first user end, cache the first multimedia stream based on a set delay time, and push the cached first multimedia stream based on the current moment;

an obtaining module, configured to, in a pushing process of the first multimedia stream, if an interaction request between a first user side and a second user side is received, obtain a second multimedia stream of the second user side;

the mixed flow module is configured to perform mixed flow processing on the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, wherein the mixed flow time for performing the mixed flow processing is less than or equal to the set delay time;

a switching module configured to switch the first multimedia stream to the mixed multimedia stream for pushing.

According to a second aspect of the embodiments of the present application, there is provided a multimedia stream switching system, including:

the first user terminal is used for sending the first multimedia stream to the server terminal;

the server is used for caching the first multimedia stream based on set delay time and pushing the cached first multimedia stream based on the current moment; in the process of pushing the first multimedia stream, if an interaction request of a first user end and a second user end is received, acquiring a second multimedia stream of the second user end; performing mixed flow treatment on the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, wherein the mixed flow time for performing mixed flow treatment is less than or equal to the set delay time; and switching the first multimedia stream into the mixed multimedia stream for pushing.

According to a fourth aspect of embodiments of the present application, there is provided a computing device comprising a memory, a processor and computer instructions stored on the memory and executable on the processor, the processor implementing the steps of the multimedia stream switching method when executing the computer instructions.

According to a fifth aspect of embodiments of the present application, there is provided a computer-readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the multimedia stream switching method.

The multimedia stream switching method receives a first multimedia stream sent by a first user terminal, caches the first multimedia stream based on a set delay time, and pushes the cached first multimedia stream based on the current moment; in the process of pushing the first multimedia stream, if an interaction request of a first user end and a second user end is received, acquiring a second multimedia stream of the second user end; performing mixed flow treatment on the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, wherein the mixed flow time for performing mixed flow treatment is less than or equal to the set delay time; and switching the first multimedia stream into the mixed multimedia stream for pushing. Therefore, the cached first multimedia stream is pushed based on the current moment, so that the first multimedia stream does not need to be cut off and pushed to different modules again, the problem of cutting off is avoided, the first multimedia stream does not need to be decoded and encoded, resources of a processor and a memory do not need to be consumed, and resource waste is avoided. By setting the delay time, the mixing time of the mixing processing is less than or equal to the set delay time, the multimedia contents corresponding to the output mixed multimedia stream can be completely aligned and smooth without a pause phenomenon, and the smoothness of multimedia switching is improved.

Drawings

FIG. 1A is a schematic diagram of a live broadcast process in the prior art;

FIG. 1B is a schematic diagram of a client-side mixing wheat-connecting process in the prior art;

FIG. 1C is a schematic diagram of a mixed flow process of a service end in the prior art;

fig. 2 is a flowchart of a multimedia stream switching method according to an embodiment of the present application;

fig. 3A is a schematic flowchart illustrating a first multimedia stream pushing method in a multimedia stream switching method according to an embodiment of the present application;

fig. 3B is a schematic flowchart illustrating a process of pushing a first multimedia stream in another multimedia stream switching method according to an embodiment of the application;

fig. 3C is a schematic flowchart illustrating a method for pushing a hybrid multimedia stream according to an embodiment of the present application;

fig. 3D is a schematic flowchart illustrating a method for pushing a hybrid multimedia stream according to another embodiment of the present application;

fig. 3E is a schematic diagram of a first multimedia stream carrying frame numbers in a multimedia stream switching method according to an embodiment of the present application;

fig. 3F is a schematic diagram of a handover procedure in a multimedia stream handover method according to an embodiment of the present application;

fig. 4 is a processing flow chart of a multimedia stream switching method applied to live broadcast and live broadcast in an embodiment of the present application;

fig. 5 is a schematic structural diagram of a multimedia stream switching apparatus according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of a multimedia stream switching system according to an embodiment of the present application;

fig. 7 is a block diagram of a computing device according to an embodiment of the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.

The terminology used in the one or more embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of the present application. As used in one or more embodiments of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present application refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments of the present application to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first aspect may be termed a second aspect, and, similarly, a second aspect may be termed a first aspect, without departing from the scope of one or more embodiments of the present application. The word "if" as used herein may be interpreted as "at" \8230; "or" when 8230; \8230; "or" in response to a determination ", depending on the context.

First, the noun terms to which one or more embodiments of the present application relate are explained.

The anchor: the viewer of the user who initiates the live broadcast will see the pictures and sound of the live broadcast from the live broadcast room of the main broadcast. Or hear the sound of its live broadcast.

A wheat connecting user: when the user is connected with the live broadcast, other audiences can watch the picture and the sound of the connected-with-the-TV user or hear the sound of the connected-with-the-TV user; the user can be a main broadcasting before connecting with the microphone or a common audience; may be one person or may be multiple persons.

Multimedia streaming: i.e. a stream, a video stream or an audio stream, usually contains both video pictures and audio sounds, but may also contain only video pictures or only audio sounds.

Live streaming: streams distributed to live rooms so that viewers can actually see them; in the non-microphone-connected state, the same as the main broadcast stream, namely the live broadcast picture and sound or sound of the main broadcast; in the connected-to-wheat state, the mixed flow is formed, and the mixed flow simultaneously contains the pictures and the sounds of the main broadcasting user and the connected-to-wheat user or the sounds.

Mixing flow: and mixing the multi-path multimedia streams of the main broadcasting user and the microphone connecting user together to form a path of multimedia stream, and distributing the path of multimedia stream to a live broadcasting room as a live broadcasting stream for the audience to watch.

In the present application, a multimedia stream switching method is provided, and the present application relates to a multimedia stream switching apparatus, a multimedia stream switching system, a computing device, and a computer-readable storage medium, which are described in detail in the following embodiments one by one.

With the rapid development of computer technology, internet technology is becoming more mature, and among them, live broadcast technology is the most prominent. Referring to fig. 1A, fig. 1A shows a schematic view of a live broadcast process in the prior art, in the live broadcast process of an anchor, a client corresponding to the anchor sends an anchor stream to a server, and the server directly distributes the anchor stream to a live broadcast room, that is, a client corresponding to a viewer. In order to enhance the interactivity of live broadcast, live broadcast and live wheat are carried out at the same time.

In the process of live broadcasting of the main broadcast, the main broadcast and the opposite party participating in the live broadcasting need to mix the pictures and the sounds of the main broadcast and the opposite party. Referring to fig. 1B, fig. 1B shows a schematic diagram of a wheat-connecting process of mixed flow at a client in the prior art, when a main broadcast and a wheat-connecting user connect a wheat, a mixed flow module is added at the client corresponding to the main broadcast and receives a wheat-connecting user flow sent by the wheat-connecting user side corresponding to the wheat-connecting user, the main broadcast flow and the wheat-connecting user flow are mixed at the client corresponding to the main broadcast to obtain a mixed live broadcast flow, and the mixed flow is sent to a server by the client corresponding to the main broadcast and is directly distributed to a live broadcast room, that is, the client corresponding to an audience. However, the mixed flow at the client is limited by the client performance corresponding to the anchor, and in the mixed flow or the wheat-connecting process for multi-person, the client corresponding to the anchor needs to run other tasks, such as playing games, superimposing special effects, etc., which are difficult to be supported by the client performance.

In order to reduce resource consumption of a client corresponding to a anchor and fully utilize the performance of a server to support multi-user mixed flow, a server mixed flow mode is generally adopted in the prior art. Referring to fig. 1C, fig. 1C shows a schematic diagram of a wheat connecting process of mixed flow at a service end in the prior art: when a main broadcast and a wheat connecting user carry out wheat connection, a mixed flow module is added at a server side and receives a wheat connecting user flow sent by a wheat connecting user side corresponding to the wheat connecting user, the main broadcast flow corresponding to the main broadcast and the wheat connecting user flow corresponding to the wheat connecting user are mixed by the server side to obtain a mixed live broadcast flow, the mixed live broadcast flow is distributed to a live broadcast room, namely the server side needs to mix the main broadcast and the live broadcast flows of the participating wheat connecting users together and then pushes the mixed live broadcast flow to an audience client side watching the live broadcast of each wheat connecting client side. Therefore, the live content of the user microphone connection interaction of each microphone connection client can be displayed at the audience client.

However, since the server needs to mix live streams of the anchor and the live stream of the connected-to-live user, it takes a certain time to directly switch the multimedia stream corresponding to the anchor to the live stream after the server mixes, which may cause the stream to be broken or the screen to be paused and then restored, thereby reducing the experience during the connected-to-live.

Therefore, the present specification provides a multimedia stream switching method, which receives a first multimedia stream sent by a first user, caches the first multimedia stream based on a set delay time, and pushes the cached first multimedia stream based on a current time; in the process of pushing the first multimedia stream, if an interaction request of a first user end and a second user end is received, acquiring a second multimedia stream of the second user end; performing mixed flow treatment on the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, wherein the mixed flow time for performing the mixed flow treatment is less than or equal to the set delay time; and switching the first multimedia stream into the mixed multimedia stream for pushing. Therefore, the cached first multimedia stream is pushed based on the current moment, so that the first multimedia stream does not need to be cut off and pushed to different modules again, the problem of cutting off is avoided, the first multimedia stream does not need to be decoded and coded, resources of a processor and a memory are not consumed, and resource waste is avoided. By setting the delay time, the mixing time of the mixing processing is less than or equal to the set delay time, the multimedia contents corresponding to the output mixed multimedia stream can be ensured to be completely aligned and smooth without pause phenomenon, and the smoothness of multimedia switching is improved.

The multimedia stream switching method provided by one or more embodiments of the present specification can be applied to multimedia stream switching in various scenarios: the online video conference, live broadcasting and microphone connecting, online voice conference, live broadcasting PK, and the like are specifically selected according to actual conditions, and this is not limited in this embodiment of the specification.

Fig. 2 shows a flowchart of a multimedia stream switching method according to an embodiment of the present application, which specifically includes the following steps:

step 202: the method comprises the steps of receiving a first multimedia stream sent by a first user terminal, caching the first multimedia stream based on set delay time, and pushing the cached first multimedia stream based on the current moment.

Specifically, the user side may be any electronic product capable of performing human-Computer interaction with a user, for example, a PC (Personal Computer), a mobile phone, a Pocket PC (Pocket PC), a tablet PC, and the like. The first user side refers to any user side which does not perform interaction. The server may be one server, or a server cluster formed by multiple servers, or a cloud computing service center, which is not limited in this embodiment of the present application. The set delay time refers to the time for delaying sending of the first multimedia stream set at the server, for example, the set delay time is 3 seconds, the server pushes the first multimedia stream received at the 1 st second at the 4 th second, and pushes the first multimedia stream received at the 2 nd second at the 5 th second; in addition, the setting of the delay time may also be understood as setting a buffering time length, that is, a time length during which the first multimedia stream that the server needs to buffer can be played.

In practical application, a user carries out live broadcasting, meeting and the like through a first user side, correspondingly, the first user side establishes a link with a server side, and sends a generated first multimedia stream to the server side, namely, a stream pulling process. Further, when receiving the first multimedia stream, the server caches the continuous first multimedia stream based on the set delay time, and pushes the cached first multimedia stream based on the current time, that is, a stream pushing process.

With reference to fig. 3A and fig. 3B, fig. 3A is a schematic flowchart of pushing a first multimedia stream in a multimedia stream switching method according to an embodiment of the present application, and fig. 3B is a schematic flowchart of pushing the first multimedia stream in another multimedia stream switching method according to an embodiment of the present application. When a first user side pushes a stream, a first multimedia stream is pushed to a mixed flow module of a server side, the mixed flow module defaults to use a copy (copy) mode, namely, only a single path of multimedia stream is input without decoding and encoding, and the input first multimedia stream is directly copied and forwarded, so that compared with the action of decoding and encoding, the resources of a Central Processing Unit (CPU) and a memory are hardly consumed; at this time, the switching unit in the mixed flow module also performs pure transparent transmission on the first multimedia stream in the buffer unit and forwards the first multimedia stream to a live broadcast room, namely, a viewer side, and does not need to do anything.

In one or more alternative embodiments of the present specification, a buffering unit is disposed in the server, and the buffering unit may buffer the first multimedia stream with the set delay time. When the server side receives the first multimedia stream, the first multimedia stream is stored through the cache unit, when the cache unit stores the first multimedia stream with enough set delay time and the server side continuously receives the first multimedia stream, the first multimedia stream cached first is pushed according to the first-in first-out rule, and the first multimedia stream received later is added to the cache unit.

For example, the delay time is set to 2 seconds, that is, the cache unit may store a first multimedia stream for 2 seconds, the server starts to receive the first multimedia stream in the 1 st second, and caches the first multimedia stream in the cache unit for the 1 st second without pushing the first multimedia stream; at the 2 nd second, the first multimedia stream of the 2 nd second is cached in a cache unit and is not pushed; when the time is 3 seconds, pushing the first multimedia stream of the 1 st second buffered in the buffer unit, and buffering the first multimedia stream of the 3 rd second in the buffer unit; and at the 4 th second, pushing the first multimedia stream of the 2 nd second buffered in the buffer unit, buffering the first multimedia stream of the 4 th second in the buffer unit, and so on.

In one or more optional embodiments of the present specification, a cache unit is disposed in the server, and the cache unit may cache the first multimedia stream with a set push time, where the set push time is greater than or equal to the set delay time. When the server receives the first multimedia stream, the first multimedia stream is stored through the cache unit, and when the cache time of the first multimedia stream in the cache unit reaches the set push time, the server takes out and pushes the first multimedia stream of which the cache time reaches the set push time from the cache unit.

Optionally, in order to guarantee the push timeliness of the first multimedia stream, the first multimedia stream may be pushed according to the current time. That is, the first multimedia stream cached based on the current time is pushed, and the specific implementation process may be as follows:

subtracting the set delay time from the current time to obtain a time to be pushed;

In practical application, real-time, that is, current time, can be obtained, the current time is subtracted from the set delay time to obtain a time to be pushed, and then the first multimedia stream corresponding to the time to be pushed is pushed. Therefore, the current time is subtracted from the set delay time to determine the time to be pushed, the method is simple and rapid, the pushing timeliness of the first multimedia stream can be guaranteed, meanwhile, the calculation is simplified, the calculation amount is reduced, the data processing pressure of the server side is reduced, and the switching efficiency of the multimedia streams is improved.

For example, the current time is 00:10, setting the delay time to be 5 seconds, and setting the time to be pushed to be 00:05, the server will 00:05 push the received first multimedia stream.

Optionally, pushing the cached first multimedia stream based on the current time may further be: subtracting the set delay time from the current time to obtain an initial time to be pushed; and subtracting the set interval duration from the initial to-be-pushed time to obtain a target to-be-pushed time, and pushing the first multimedia stream corresponding to the target to-be-pushed time.

For example, the current time is 00:10, setting the delay time to be 1 second, setting the interval duration to be 1 second, and setting the initial push time to be 00:09, the target time to be pushed is 00:08, the server will 00:08, the received first multimedia stream.

Step 204: in the process of pushing the first multimedia stream, if an interaction request between a first user end and a second user end is received, a second multimedia stream of the second user end is obtained.

Specifically, the second user end refers to any user end that establishes interaction with the first user end. The interaction request refers to a request initiated by two users through a client, such as a request for connecting to a microphone, a request for participating in an online conference, and the like, where the request for initiating the interaction may be a first client or a second client, and the application is not limited to this.

In practical application, in the process of pushing the first multimedia stream, if an interaction request between the first user end and the second user end is received, the server end obtains the second multimedia stream of the second user end, which can enable the second user end to obtain the second multimedia stream, and can also obtain the second multimedia stream from the server end that pushes the stream from the second user end, which is not limited in this application.

After receiving the interaction request, the server side can acquire a second multimedia stream of a second user side in real time; the interaction request may also carry a request timestamp, and the server may obtain the request timestamp and a second multimedia stream after the request timestamp based on the request timestamp.

It should be noted that, in order to ensure the correctness of the mixed flow, before the second multimedia stream of the second user end is obtained, it is also required to verify whether the interaction request between the first user end and the second user end passes through, if so, the second multimedia stream of the second user end is obtained, and if not, the second multimedia stream of the second user end is not obtained. For example, a video conference initiator invites a person to join a video conference through a first client, if the person refuses the invitation of the video conference initiator through a second client, a second multimedia stream of the second client is not obtained, and if the person agrees the invitation of the video conference initiator through the second client, the second multimedia stream of the second client is obtained.

Step 206: and performing mixed flow treatment on the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, wherein the mixed flow time for performing the mixed flow treatment is less than or equal to the set delay time.

In practical applications, after the first multimedia stream and the second multimedia stream are obtained, the first multimedia stream and the second multimedia stream need to be mixed, so as to obtain a mixed multimedia stream. It should be noted that the mixing time for mixing is less than or equal to the set delay time.

In one or more alternative embodiments of the present disclosure, the multimedia stream is composed of a plurality of multimedia frames carrying time stamps, and in order to improve the quality of the mixed multimedia stream, the multimedia streams with the same time stamps need to be mixed. That is to say, under the condition that the first multimedia stream is composed of a plurality of first multimedia frames carrying timestamps and the second multimedia stream is composed of a plurality of second multimedia frames carrying timestamps, the mixed stream is obtained by performing mixed stream processing on the first multimedia stream and the second multimedia stream, and the specific implementation process may be as follows:

and mixing the first multimedia frame and the second multimedia frame carrying the same timestamp to obtain a mixed multimedia frame carrying the timestamp, wherein the mixed multimedia frames form a mixed multimedia stream.

In practical applications, each first multimedia frame in the first multimedia stream and each second multimedia frame in the second multimedia stream carry a time stamp. The essence of the blending process is to blend the first multimedia frame with the second multimedia frame. In order to ensure the authenticity and accuracy of the mixed multimedia frame, the timestamps of the first multimedia frame and the second multimedia frame need to be compared, and the first multimedia frame and the second multimedia frame carrying the same timestamp are mixed to obtain the mixed multimedia frame also carrying the same timestamp, so as to obtain the mixed multimedia stream.

In one or more alternative embodiments of the present disclosure, the first multimedia stream and the second multimedia stream may be decoded, mixed, and encoded to obtain a mixed multimedia stream. That is, the mixed flow processing is performed on the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, and the specific implementation process may be as follows:

In practical applications, when performing the mixed flow processing, the first multimedia stream may be decoded to obtain a decoded first multimedia stream, and the second multimedia stream may be decoded to obtain a decoded second multimedia stream. And then mixing the decoded first multimedia stream and the decoded second multimedia stream to obtain an initial mixed multimedia stream. And further, coding the initial mixed multimedia stream to obtain a target mixed multimedia stream.

Therefore, the first multimedia stream and the second multimedia stream are decoded, so that the first multimedia stream and the second multimedia stream can be mixed more easily, and the mixed multimedia stream is more accurate.

On the basis of fig. 3B, the description is made with reference to fig. 3C and fig. 3D, where fig. 3C shows a schematic flow chart of pushing a mixed multimedia stream in a multimedia stream switching method provided by an embodiment of the present application, and fig. 3D shows a schematic flow chart of pushing a mixed multimedia stream in another multimedia stream switching method provided by an embodiment of the present application: when the first user terminal and the second user terminal interact with each other, namely the server terminal receives an interaction request of the first user terminal and the second user terminal, the second multimedia of the second user terminal is also directly pushed to the mixed flow module of the server terminal. At this time, the mixed flow module has more than one path of multimedia stream input, the mixed flow mode is automatically started, namely, the two decoding units respectively decode the first multimedia stream and the second multimedia stream, then the mixing unit performs mixing processing, and the coding unit performs coding processing to obtain the mixed multimedia stream.

Because extra time delay (typically 1-2 s) is brought by decoding, mixed flow and encoding actions, namely mixed flow time length, the output of the mixed flow module is still discontinuous when the mixed flow module is directly switched from a copy mode to a mixed flow mode; therefore, in the copy mode, the buffer unit will buffer a small segment of data (typically 1-2 s) according to the corresponding delay, i.e. buffer the first multimedia stream with the set delay push time; when the copy mode is switched to the mixed flow mode, the output data still comes from the first multimedia flow cached by the cache unit until the coding unit after mixed flow starts to output, so that the output can be smooth and continuous.

Step 208: and switching the first multimedia stream into the mixed multimedia stream for pushing.

In practical applications, after the mixed multimedia stream is obtained, the first multimedia stream can be switched to the mixed multimedia stream for pushing. Referring to fig. 3C and fig. 3D, the mixed multimedia stream is pushed to the live broadcasting room by the mixed stream module, specifically, the encoding unit performs encoding processing, and after the mixed multimedia stream is obtained, the switching module switches the first multimedia stream into the mixed multimedia stream and pushes the mixed multimedia stream to the live broadcasting room.

In one or more alternative embodiments of the present disclosure, in order to ensure continuity of pushing and further improve the user enjoyment, after the mixed multimedia stream is obtained, a request timestamp carried in the interaction request may be added to a set delay time to obtain a switching time, and then, when the switching time is reached, the first multimedia stream is switched to the mixed multimedia stream for pushing.

For example, the request timestamp is 10:00, setting the delay time to be 5 seconds, and then setting the switching time to be 10:05, i.e. between 10: and 05, switching the first multimedia stream into a mixed multimedia stream for pushing.

In one or more optional embodiments of the present specification, the media stream is composed of a plurality of multimedia frames with timestamps, and in order to ensure consistency of pushing and further improve user enjoyment, the timestamp carried by the first multimedia frame to be pushed that can be cached is compared with the timestamp carried by the mixed multimedia frame, and whether to switch is determined according to the comparison result. That is, in the case that the first multimedia stream is composed of a plurality of first multimedia frames carrying timestamps, and the mixed multimedia stream is composed of a plurality of mixed multimedia frames carrying timestamps, the first multimedia stream is switched to the mixed multimedia stream for pushing, and the specific implementation process may be as follows:

if yes, the cached first multimedia frame to be pushed is switched to the target mixed multimedia frame for pushing.

In practical applications, each multimedia frame carries a time stamp. For the first multimedia frame to be pushed, namely the first multimedia frame which is cached and is to be pushed, the timestamp carried by the first multimedia frame is compared with the timestamp carried by the mixed multimedia frame. And if the target mixed multimedia frames with the same timestamp exist, replacing the first multimedia frame with the target mixed multimedia frame for pushing, namely switching the target mixed multimedia frame into the target mixed multimedia frame for pushing, and then sequentially pushing the mixed multimedia frames according to the sequence of the timestamps from front to back. If the target mixed multimedia frame with the same timestamp does not exist, the first multimedia frame is pushed, and the process is repeated when the first multimedia frame which is cached next and is to be pushed exists.

Therefore, the multimedia stream pushed to the audience end is continuous and connected front to back, the problem that the multimedia stream is blocked or jumped is avoided, and the switching efficiency and the user viscosity are improved.

Optionally, on the basis of determining the target mixed multimedia frame, the first multimedia frame which is cached and is to be pushed may be directly switched to the target mixed multimedia frame for pushing, so that the switching efficiency is improved, the switching flow is slowed down, and the processing efficiency of the server is improved.

Optionally, on the basis of determining the target mixed multimedia frame, it may be further determined whether the target mixed multimedia frame or the first multimedia frame is a key frame. And determining whether to switch according to the judgment result. That is, the first multimedia frame carries identification information, where the identification information indicates whether the first multimedia frame is a key frame, and at this time, the cached and to-be-pushed first multimedia frame is switched to the target mixed multimedia frame for pushing, and a specific implementation process may be as follows:

In practical application, the first multimedia frame carries identification information. After the target mixed multimedia frame is determined, whether the cached first multimedia frame to be pushed is a key frame or not can be identified, that is, identification information carried by the cached first multimedia frame to be pushed is identified. If yes, the target mixed multimedia frame is switched to the target mixed multimedia frame to be pushed, and then all mixed multimedia frames are pushed in sequence according to the sequence of the timestamps from front to back. If not, pushing the first cached to-be-pushed multimedia frame, and repeating the process when the first cached to-be-pushed multimedia frame is next.

In addition, since the mixed multimedia frame is obtained by mixing the first multimedia frame and the second multimedia frame, the mixed multimedia frame also carries the identification information. Switching the cached first multimedia frame to be pushed into the target mixed multimedia frame for pushing, or identifying whether the mixed multimedia frame is a key frame according to identification information carried by the target mixed multimedia frame; if so, switching the cached first multimedia frame to be pushed into the target mixed multimedia frame for pushing; if not, pushing the cached first multimedia frame to be pushed.

Due to the fact that the non-key frames can have forward dependence, even if the timestamps are the same, the problems of screen splash and the like can be caused when the non-key frames are directly used, the problems of screen splash and the like can be avoided when switching is conducted based on the judgment result of the key frames, the accuracy and the efficiency of multimedia stream switching are further improved, and the appreciation degree and the viscosity of users are improved.

It should be noted that if the interaction between the first user end and the second user end is disconnected, the above-mentioned reverse operation of switching the multimedia stream can be executed. That is, after the switching the first multimedia stream into the hybrid multimedia stream for pushing, the method further includes: and if the interaction disconnection between the first user terminal and the second user terminal is detected, switching the mixed multimedia stream into the first multimedia stream for pushing.

For example, referring to fig. 3E, fig. 3E shows a schematic diagram of a first multimedia stream carrying frame numbers in a multimedia stream switching method according to an embodiment of the present application: the first multimedia stream input by the mixed stream module can be numbered for each first multimedia frame; the numbering method is not limited to continuously increasing numerical values or display time stamps, etc., preferably the labels are time stamps; and recording whether each frame is a key frame, namely, the first multimedia frame carries identification information.

In a copy mode, caching a first multimedia stream with set delay time in a caching unit, wherein the frame number of the cached first multimedia frame can be determined by the difference value of a timestamp carried by a second multimedia frame, or the first multimedia stream is obtained by dividing the set delay time by a frame rate (the number of frames per second); the set delay time is always ensured to be larger than the delay caused by decoding, mixing and encoding, namely the mixed flow time of mixed flow treatment is smaller than or equal to the set delay time;

when the mixed flow mode is started, the first multimedia frames with the same number are sent to the decoding unit, the mixing unit and the coding unit module, and each module reserves a timestamp carried by the first multimedia frame;

referring to fig. 3F, fig. 3F is a schematic diagram illustrating a handover procedure in a multimedia stream handover method according to an embodiment of the present application: after the mixed stream mode encoding unit starts outputting, the switching unit receives a mixed multimedia stream, wherein each mixed multimedia frame of the mixed multimedia stream has a time stamp (e.g., 1,2, \ 8230;, N) and identification information. At this time, the switching unit enters the switching logic: first buffering all data from the mixed multimedia stream (mixed multimedia frames) in the switching unit; then, before sending a frame of first multimedia frame (multimedia frame in the first multimedia stream from the buffer unit), the switching unit always searches whether a mixed multimedia frame with the same timestamp as the first multimedia frame exists in the mixed multimedia frames of the mixed multimedia stream, if not, the next frame continues to search; if the key frame exists, the aligned mixed multimedia frames, such as the first cache frame 2 and the mixed multimedia frame 2, are found, and then switching is performed at the position of the next mixed multimedia frame or the first video frame, which carries the identification information of the key frame, namely switching is performed at the next key frame; after switching, the expired buffer (the first multimedia frame) before the mixed flow multimedia frame can be cleaned; the multimedia stream then completes the seamless handover process.

It should be noted that the multimedia frames may be audio frames or video frames, and the audio frames and the video frames may follow the switching logic. Furthermore, audio frames can each be considered as key frames, so as long as aligned timestamps are found, a switch can be made. If the video frame is a video frame, in order to ensure the synchronization of the sound and the picture, the actual switching process should occur when the next switching point, namely the next key frame after picture alignment and the key frame at the same position after sound alignment are found, and switching is performed simultaneously, so as to ensure the synchronization of the sound and the picture;

after switching, the timestamps of the first multimedia stream and the mixed multimedia stream from the buffer unit are always aligned, and the first multimedia stream and the mixed multimedia stream can be switched back to the copy mode at any time, such as abnormal or interaction ending; in the interaction process, the first multimedia stream from the buffer unit enters the switching unit to be directly discarded without additional processing.

Optionally, bushy of multimedia frames may only number key frames, for example, one key frame every 1 to 10 seconds during live broadcasting, so when querying and aligning in the switching module, only querying and aligning for key frames is needed, and frames of non-key frames (without numbers) are skipped, which may greatly optimize querying efficiency during aligning.

When the interaction is finished, the second multimedia stream is disconnected, the input of the mixed flow module is reduced to a return flow, namely the first multimedia stream, and the mixed flow mode is switched to the copy mode by following the reverse operation, so that the resources do not need to be consumed additionally. The mixed flow module can fully utilize server resources and conveniently support multi-user interaction.

Therefore, when the anchor is broadcast directly, the anchor stream always passes through the optimized mixed flow module, so that the anchor stream does not need to be cut off and then pushed to different modules, and the problem of cut-off is solved; the optimized mixed flow module is in a copy mode by default, namely decoding and encoding are not carried out when only one-way stream is input, and the input stream is directly copied and forwarded, so that compared with the decoding and encoding actions, the resources of a CPU and a memory are hardly consumed, and obvious resource waste is avoided. When the copy mode and the mixed flow mode are switched in the mixed flow module, seamless flow switching can be realized through cache and frame number alignment, namely, the output audio and video contents are completely aligned and smooth without a pause phenomenon.

The multimedia stream switching method receives a first multimedia stream sent by a first user terminal, caches the first multimedia stream based on a set delay time, and pushes the cached first multimedia stream based on the current moment; in the process of pushing the first multimedia stream, if an interaction request of a first user end and a second user end is received, acquiring a second multimedia stream of the second user end; performing mixed flow treatment on the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, wherein the mixed flow time for performing the mixed flow treatment is less than or equal to the set delay time; and switching the first multimedia stream into the mixed multimedia stream for pushing. Therefore, the cached first multimedia stream is pushed based on the current moment, so that the first multimedia stream does not need to be cut off and pushed to different modules again, the problem of cutting off is avoided, the first multimedia stream does not need to be decoded and coded, resources of a processor and a memory are not consumed, and resource waste is avoided. By setting the delay time, the mixing time of the mixing processing is less than or equal to the set delay time, the multimedia contents corresponding to the output mixed multimedia stream can be ensured to be completely aligned and smooth without pause phenomenon, and the smoothness of multimedia switching is improved.

The following describes the multimedia stream method further by taking an application of the multimedia stream switching method provided in the present application in live broadcasting and microphone connecting as an example, with reference to fig. 4. Fig. 4 shows a processing flow chart of a multimedia stream switching method applied to live broadcast and live broadcast in an embodiment of the present application, which specifically includes the following steps:

step 402: the method comprises the steps of receiving a first video stream sent by an anchor terminal, and caching the first video stream based on set delay time, wherein the first video stream is composed of a plurality of first video frames carrying timestamps.

Step 404: subtracting the set delay time from the current time to obtain the time to be pushed; and pushing the first video frame corresponding to the moment to be pushed.

Step 406: in the process of pushing the first video stream, if an interaction request between the anchor terminal and the connected-to-microphone client terminal is received, a second video stream of the connected-to-microphone client terminal is obtained, wherein the second video stream is composed of a plurality of second video frames carrying timestamps.

Step 408: and decoding the first video frame and the second video frame which carry the same time stamp to obtain the decoded first video frame and the decoded second video frame.

Step 410: mixing the decoded first video stream and the decoded second video stream to obtain an initial mixed video stream; and coding the initial mixed video stream to obtain a target mixed video stream, wherein a plurality of mixed video frames form the mixed video stream.

Step 412: and comparing the cached first video frame to be pushed with the obtained mixed video frame, and checking whether a target mixed video frame with the same timestamp as the timestamp carried by the first video frame to be pushed exists.

Step 414: if the first video frame exists, whether the first video frame which is cached and is to be pushed is a key frame is identified according to the identification information carried by the first video frame which is cached and is to be pushed.

Step 416: if so, switching the cached first video frame to be pushed into the target mixed video frame for pushing.

Step 418: and if the interaction disconnection between the main broadcasting end and the microphone connecting user end is detected, switching the mixed video stream into a first video frame for pushing.

According to the multimedia stream switching method, the cached first multimedia stream is pushed based on the current moment, so that the first multimedia stream does not need to be pushed to different modules again in a flow interruption mode, the flow interruption problem is avoided, the first multimedia stream does not need to be decoded and coded, resources of a processor and a memory do not need to be consumed, and resource waste is avoided. By setting the delay time, the mixing time of the mixing processing is less than or equal to the set delay time, the multimedia contents corresponding to the output mixed multimedia stream can be ensured to be completely aligned and smooth without pause phenomenon, and the smoothness of multimedia switching is improved.

Corresponding to the foregoing method embodiment, the present application further provides an embodiment of a multimedia stream switching apparatus, and fig. 5 shows a schematic structural diagram of a multimedia stream switching apparatus provided in an embodiment of the present application. As shown in fig. 5, the apparatus includes:

a receiving module 502, configured to receive a first multimedia stream sent by a first user, buffer the first multimedia stream based on a set delay time, and push the buffered first multimedia stream based on a current time;

an obtaining module 504, configured to, in a pushing process of the first multimedia stream, if an interaction request between a first user side and a second user side is received, obtain a second multimedia stream of the second user side;

a mixing module 506 configured to mix the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, wherein the mixing time for mixing is less than or equal to the set delay time;

a switching module 508 configured to switch the first multimedia stream to the mixed multimedia stream for pushing.

Optionally, the switching module 508 is further configured to:

the flow mixing module 506, further configured to:

Optionally, the first multimedia stream is composed of a plurality of first multimedia frames carrying timestamps, and the hybrid multimedia stream is composed of a plurality of hybrid multimedia frames carrying timestamps;

the switching module 508, further configured to:

Optionally, the flow mixing module 506 is further configured to:

Optionally, the switching module 508 is further configured to:

and if the interaction disconnection between the first user side and the second user side is detected, switching the mixed multimedia stream into the first multimedia stream for pushing.

According to the multimedia stream switching device, the cached first multimedia stream is pushed based on the current moment, so that the first multimedia stream does not need to be cut off and pushed to different modules, the problem of flow cutoff is avoided, the first multimedia stream does not need to be decoded and coded, resources of a processor and a memory do not need to be consumed, and resource waste is avoided. By setting the delay time, the mixing time of the mixing processing is less than or equal to the set delay time, the multimedia contents corresponding to the output mixed multimedia stream can be completely aligned and smooth without a pause phenomenon, and the smoothness of multimedia switching is improved.

The foregoing is a schematic diagram of a multimedia stream switching apparatus according to the embodiment. It should be noted that the technical solution of the multimedia stream switching apparatus and the technical solution of the multimedia stream switching method belong to the same concept, and details that are not described in detail in the technical solution of the multimedia stream switching apparatus can be referred to the description of the technical solution of the multimedia stream switching method.

Corresponding to the foregoing method embodiment, the present application further provides an embodiment of a multimedia stream switching system, and fig. 6 illustrates a schematic structural diagram of a multimedia stream switching system provided in an embodiment of the present application. As shown in fig. 6, the system includes:

a first user side 602, a server side 604, a second user side 606, and a viewer side 608;

a first user terminal 602, configured to send a first multimedia stream to a server terminal 604;

the server 606 is configured to cache the first multimedia stream based on a set delay time, and push the cached first multimedia stream based on a current time; in the process of pushing the first multimedia stream, if an interaction request of a first user end and a second user end is received, acquiring a second multimedia stream of the second user end; performing mixed flow treatment on the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, wherein the mixed flow time for performing the mixed flow treatment is less than or equal to the set delay time; and switching the first multimedia stream into the mixed multimedia stream to push the viewer end 608.

Optionally, the server 606 is further configured to:

the server 606 is further configured to:

Optionally, the server 606 is further configured to:

The multimedia stream switching system provided by the application pushes the cached first multimedia stream based on the current moment, so that the first multimedia stream does not need to be cut off and pushed to different modules again, the problem of cutting off is avoided, the first multimedia stream does not need to be decoded and encoded, resources of a processor and a memory do not need to be consumed, and resource waste is avoided. By setting the delay time, the mixing time of the mixing processing is less than or equal to the set delay time, the multimedia contents corresponding to the output mixed multimedia stream can be completely aligned and smooth without a pause phenomenon, and the smoothness of multimedia switching is improved.

The foregoing is a schematic solution of a multimedia stream switching system according to this embodiment. It should be noted that the technical solution of the multimedia stream switching system and the technical solution of the multimedia stream switching method belong to the same concept, and details that are not described in detail in the technical solution of the multimedia stream switching system can be referred to the description of the technical solution of the multimedia stream switching method.

Fig. 7 shows a block diagram of a computing device according to an embodiment of the present application. Components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. Processor 720 is coupled to memory 710 via bus 730, and database 750 is used to store data.

Computing device 700 also includes access device 740, access device 740 enabling computing device 700 to communicate via one or more networks 760. Examples of such networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. The Access device 740 may include one or more of any type of Network Interface (e.g., a Network Interface Controller (NIC)) whether wired or Wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) Wireless Interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) Interface, an ethernet Interface, a Universal Serial Bus (USB) Interface, a cellular Network Interface, a bluetooth Interface, a Near Field Communication (NFC) Interface, and so forth.

In one embodiment of the application, the above components of the computing device 700 and other components not shown in fig. 7 may also be connected to each other, for example, by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 7 is for purposes of example only and is not limiting as to the scope of the present application. Those skilled in the art may add or replace other components as desired.

Computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smartphone), wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 700 may also be a mobile or stationary server.

Wherein, processor 720, when executing said computer instructions, implements the steps of said multimedia stream switching method.

The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the multimedia stream switching method described above belong to the same concept, and details that are not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the multimedia stream switching method described above.

An embodiment of the present application further provides a computer readable storage medium, which stores computer instructions, and the computer instructions, when executed by a processor, implement the steps of the multimedia stream switching method as described above.

The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the above multimedia stream switching method, and for details that are not described in detail in the technical solution of the storage medium, reference may be made to the description of the technical solution of the above multimedia stream switching method.

The foregoing description of specific embodiments of the present application has been presented. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying said computer program code, a recording medium, a usb-disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a Random Access Memory (RAM), an electrical carrier signal, a telecommunications signal, a software distribution medium, etc.

It should be noted that for simplicity and convenience of description, the above-described method embodiments are described as a series of combinations of acts, but those skilled in the art will appreciate that the present application is not limited by the order of acts, as some steps may, in accordance with the present application, occur in other orders and/or concurrently. Further, those skilled in the art will appreciate that the embodiments described in this specification are presently considered to be preferred embodiments and that acts and modules are not required in the present application.

In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to the related descriptions of other embodiments.

The preferred embodiments of the present application disclosed above are intended only to aid in the explanation of the application. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the application and its practical application, to thereby enable others skilled in the art to best understand the application and its practical application. The application is limited only by the claims and their full scope and equivalents.

Claims

1. A method for switching multimedia streams, comprising:

receiving a first multimedia stream sent by a first user terminal, caching the first multimedia stream based on set delay time, and pushing the cached first multimedia stream based on the current moment;

2. The method of claim 1, wherein the pushing the buffered first multimedia stream based on the current time comprises:

3. The method of claim 1, wherein the first multimedia stream is composed of a plurality of first multimedia frames carrying time stamps, and wherein the second multimedia stream is composed of a plurality of second multimedia frames carrying time stamps;

4. The method according to claim 1 or 3, wherein the first multimedia stream is composed of a plurality of first multimedia frames carrying time stamps, and the hybrid multimedia stream is composed of a plurality of hybrid multimedia frames carrying time stamps;

5. The method of claim 4, wherein the first multimedia frame carries identification information indicating whether the first multimedia frame is a key frame;

6. The method of claim 1, wherein the mixing the first multimedia stream with the second multimedia stream to obtain a mixed multimedia stream comprises:

7. The method of claim 1, wherein after the switching the first multimedia stream into the hybrid multimedia stream for pushing, further comprising:

8. A multimedia stream switching apparatus, comprising:

the acquisition module is configured to acquire a second multimedia stream of a second user side if an interaction request of a first user side and the second user side is received in the pushing process of the first multimedia stream;

9. A multimedia stream switching system, comprising:

the server is used for caching the first multimedia stream based on set delay time and pushing the cached first multimedia stream based on the current moment; in the process of pushing the first multimedia stream, if an interaction request of a first user end and a second user end is received, acquiring a second multimedia stream of the second user end; performing mixed flow treatment on the first multimedia stream and the second multimedia stream to obtain a mixed multimedia stream, wherein the mixed flow time for performing the mixed flow treatment is less than or equal to the set delay time; and switching the first multimedia stream into the mixed multimedia stream for pushing a viewer.

10. A computing device comprising a memory, a processor, and computer instructions stored on the memory and executable on the processor, wherein the processor implements the steps of the method of any one of claims 1-7 when executing the computer instructions.

11. A computer-readable storage medium storing computer instructions, which when executed by a processor, perform the steps of the method of any one of claims 1 to 7.