CN116095411A

CN116095411A - Audio and video stream pushing method, electronic device, electronic equipment and medium

Info

Publication number: CN116095411A
Application number: CN202111307041.1A
Authority: CN
Inventors: 张晨旸; 刘竞超; 苗辉
Original assignee: Guizhou Baishancloud Technology Co Ltd
Current assignee: Guizhou Baishancloud Technology Co Ltd
Priority date: 2021-11-05
Filing date: 2021-11-05
Publication date: 2023-05-09

Abstract

The application discloses an audio and video stream pushing method, an electronic device, electronic equipment and a medium. In the application, after the push server receives the audio and video stream pushed by the push terminal with the first code rate, the network operation state of receiving the audio and video stream by the push server can be detected, push information for indicating the push code rate of pushing the audio and video stream by the push terminal next time is generated according to the network operation state, and then the push information is sent to the push terminal, and then the push terminal is received to push the audio and video stream with the second code rate determined according to the push information.

Description

Audio and video stream pushing method, electronic device, electronic equipment and medium

Technical Field

The present application relates to data processing technologies, and in particular, to a method for pushing an audio and video stream, an electronic device, an electronic apparatus, and a medium.

Background

In the related technology, in the live broadcast process, an audio and video stream is generated by a push stream end and is sent to a CDN server, so that a pull stream end pulls streams through CDN nodes to obtain the audio and video stream, and audio and video images are displayed based on the obtained audio and video stream, thereby realizing that the video images generated by the push stream end can be seen at the pull stream end.

However, in the live broadcast process, the situation of audio and video jamming is usually caused by the oscillation of the network environment where the plug-end device is located, so that the live broadcast experience is affected.

Disclosure of Invention

The embodiment of the application provides a method, an electronic device, electronic equipment and a medium for pushing an audio and video stream, and the technical scheme of the application can avoid the problem that video is blocked due to the vibration of a network environment where equipment is located in a live broadcast process in the related technology.

According to one aspect of the embodiment of the present application, a method for pushing an audio/video stream is provided, which is applied to a push server, and includes:

receiving an audio and video stream pushed by a pushing end at a first code rate;

detecting a network running state of receiving the audio and video stream, and generating push information according to the network running state, wherein the push information is used for indicating a push code rate of pushing the audio and video stream next time by the push terminal;

the plug flow information is sent to the plug flow end;

and receiving the audio and video stream pushed by the pushing end according to the second code rate determined by the pushing information.

Optionally, in another embodiment of the method according to the present application, the generating plug flow information according to the network operation state includes:

If the network running state is determined to correspond to the network light load, generating pushing information for indicating the pushing end to push at a higher rate than the last pushing; or alternatively, the first and second heat exchangers may be,

if the network running state is determined to correspond to the network normal, generating pushing information for indicating the pushing end to continue pushing with the latest pushing code rate; or alternatively, the first and second heat exchangers may be,

and if the network running state is determined to correspond to network congestion, generating pushing information for indicating the pushing end to push with the pushing code rate not higher than the last pushing code rate.

Optionally, in another embodiment of the foregoing method according to the present application, if it is determined that the network operation state corresponds to a network light load, generating push information for indicating the push end to push at a higher rate than the last push rate includes:

and if the network running state is determined to correspond to the network light load, generating the plug flow information of the plug flow which is higher than the code rate adjacent to the last plug flow code rate.

Optionally, in another embodiment of the method according to the present application, after the generating of the push information for instructing the push end to push at a higher rate than the last push rate, the method further includes:

And generating pushing information for indicating the pushing end to push at a code rate higher than and adjacent to the latest pushing code rate only when the network running state of the audio and video stream continuously received for many times is detected to correspond to the network light load after the latest pushing code rate is detected to reach the preset threshold.

Optionally, in another embodiment of the foregoing method according to the present application, if it is determined that the network operation state corresponds to network congestion, generating push information for indicating the push end to push at a rate not higher than a last push rate includes:

and if the network running state is determined to correspond to network congestion, generating pushing information for indicating the pushing end to push the audio and video stream with an initial pushing code rate, wherein the initial pushing code rate is the code rate corresponding to the pushing end pushing the audio and video stream to the pushing server for the first time.

Optionally, in another embodiment of the foregoing method according to the present application, the initial push rate is a lowest rate at which the push end pushes the audio and video stream to the push server.

Optionally, in another embodiment of the method according to the present application, after the generating push information according to the network operation state, the method further includes:

If the condition that the same network operation state is continuously received for multiple times is detected, increasing or reducing the time interval for sending the push flow information;

and sending the push information to the push end according to the time interval.

Optionally, in another embodiment of the method according to the present application, the detecting the network operation state of receiving the audio/video stream includes:

after receiving the audio and video stream, detecting at least one of a packet loss rate parameter, a transmission rate parameter and an interval jitter parameter in the current network state;

and determining the network running state of receiving the audio and video stream based on at least one of the packet loss rate parameter, the transmission rate parameter and the interval jitter parameter.

According to one aspect of the embodiment of the present application, a method for pushing an audio/video stream is provided, which is applied to a pushing end and includes:

pushing the audio and video stream to a push server at a first code rate;

receiving push information generated by the push server based on the audio and video stream;

pushing the audio and video stream to the push server by the second code rate represented by the push information.

Optionally, in another embodiment of the method according to the present application, the pushing the audio and video stream to the push server with the first code rate includes:

If the fact that the audio and video stream is pushed to the push server for the first time is determined, pushing the audio and video stream to the push server by the push end according to the lowest code rate of pushing the audio and video stream to the push server.

According to still another aspect of the embodiments of the present application, an electronic device for pushing an audio and video stream is provided, which is applied to a push server, and includes:

the first receiving module is configured to receive the audio and video stream pushed by the pushing end at the first code rate;

the first generation module is configured to detect a network operation state of receiving the audio and video stream, and generate push information according to the network operation state, wherein the push information is used for indicating a push code rate of the push end for pushing the audio and video stream next time;

the sending module is configured to send the push information to the push end;

and the second receiving module is configured to receive the audio and video stream pushed by the pushing end according to the second code rate determined by the pushing information.

According to still another aspect of the embodiments of the present application, an electronic device for pushing an audio and video stream is provided, which is applied to a pushing end, and includes:

the first pushing module is configured to push the audio and video stream to the push server at a first code rate;

The second generation module is configured to receive push information generated by the push server based on the network running state of the audio and video stream;

and the second pushing module is configured to push the audio and video stream to the push server at a second code rate characterized by the push information.

According to still another aspect of the embodiments of the present application, there is provided an electronic device including:

a memory for storing executable instructions; and

and the display is used for displaying with the memory to execute the executable instructions so as to finish the operation of any one of the methods for pushing the audio and video streams.

According to a further aspect of the embodiments of the present application, there is provided a computer program product, including a computer program, characterized in that the computer program is executed by a processor to perform any of the above methods for pushing an audio-video stream.

According to still another aspect of the embodiments of the present application, a computer-readable storage medium is provided for storing computer-readable instructions that, when executed, perform the operations of any of the methods for audio-video streaming.

In the application, after the push server receives the audio and video stream pushed by the push terminal with the first code rate, the network operation state of receiving the audio and video stream by the push server can be detected, push information for indicating the push code rate of pushing the audio and video stream by the push terminal next time is generated according to the network operation state, and then the push information is sent to the push terminal, and then the push terminal is received to push the audio and video stream with the second code rate determined according to the push information. By applying the technical scheme, the push server can pointedly generate push information for informing the push end of pushing the push code rate matched with the network operation condition next time according to the network operation condition of the audio and video stream pushed by the push end currently, so that the frequency and time consumption of code stream switching caused by the fact that the audio and video stream is blocked in an oscillation network can be reduced, the low delay of streaming media transmission is further ensured, and the problem that the audio and video is blocked due to the oscillation of the network environment where the push end equipment is located in the live broadcast process of a user is avoided.

The technical scheme of the present application is described in further detail below through the accompanying drawings and examples.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and, together with the description, serve to explain the principles of the application.

The present application will be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

fig. 1 is a schematic diagram of a method for pushing an audio/video stream according to the present application;

fig. 2 is a schematic diagram of another method for pushing an audio/video stream according to the present application;

fig. 3 is a flow chart of a method for pushing an audio/video stream according to the present application;

fig. 4-5 are schematic diagrams of an electronic device for pushing an audio/video stream according to the present application;

fig. 6 is a schematic diagram of another electronic device for pushing an audio/video stream according to the present application.

Detailed Description

Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present application unless it is specifically stated otherwise.

Meanwhile, it should be understood that the sizes of the respective parts shown in the drawings are not drawn in actual scale for convenience of description.

The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the application, its application, or uses.

Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail, but are intended to be part of the specification where appropriate.

It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further discussion thereof is necessary in subsequent figures.

In addition, the technical solutions of the embodiments of the present application may be combined with each other, but it is necessary to be based on the fact that those skilled in the art can implement the technical solutions, and when the technical solutions are contradictory or cannot be implemented, the combination of the technical solutions should be considered to be absent, and is not within the scope of protection claimed in the present application.

It should be noted that all directional indicators (such as up, down, left, right, front, and rear … …) in the embodiments of the present application are merely used to explain the relative positional relationship, movement conditions, and the like between the components in a specific posture (as shown in the drawings), and if the specific posture is changed, the directional indicator is correspondingly changed.

A method for performing audio and video stream pushing according to an exemplary embodiment of the present application is described below with reference to fig. 1 to 3. It should be noted that the following application scenario is only shown for the convenience of understanding the spirit and principles of the present application, and embodiments of the present application are not limited in any way in this respect. Rather, embodiments of the present application may be applied to any scenario where applicable.

In an embodiment, the present application further provides a method for pushing an audio/video stream. Fig. 1 schematically shows a flow diagram of a method for pushing an audio/video stream according to an embodiment of the present application. As shown in fig. 1, the method is applied to a push server, and includes:

s101, receiving an audio/video stream pushed by a pushing end at a first code rate.

In one mode, the push server mentioned in the application may be an edge server of a content delivery network (content delivery network, CDN), an area server, or a central server. It should be noted that, when the push server is an edge server, the push end may be a client used by a user. When the push server is a regional server or a central server, the push end may be an edge server in communication with the regional server or a regional server in communication with the central server.

In one mode, the first code rate may be a code rate used when the push end pushes the audio and video stream to the push server for the first time. In this case, the first code rate may be the lowest code rate that the push end can reach in the current network situation, or may be the next lowest code rate that the push end can reach in the current network situation, or the like.

S102, detecting a network operation state of receiving the audio and video stream, and generating push information according to the network operation state, wherein the push information is used for indicating a push code rate of pushing the audio and video stream next time by a push terminal.

After receiving the audio/video stream with the first code rate pushed by the pushing end, the pushing server in the embodiment of the application can dynamically generate the pushing information for indicating the pushing code rate of the next pushing audio/video stream by the pushing end in real time according to the current network running state.

It can be understood that the push server can correspondingly generate corresponding push information according to whether the network of the current received audio and video stream is smooth or not. The pushing information may be pushing information for indicating that the pushing end performs pushing with a higher rate than the last pushing rate, pushing information for indicating that the pushing end continues pushing with the last pushing rate, pushing information for indicating that the pushing end performs pushing with a rate not higher than the last pushing rate, and so on.

Specifically, when the push server sends push information to the push end, the push end needs to push at a higher rate than the last push rate after receiving the push information (e.g., push from 500Kbps to 1000 Kbps). And when the push server sends push information for indicating the push end to continue pushing with the last push code rate to the push end, the push end needs to continue pushing with the last push code rate after receiving the push information (for example, maintaining the push to the push server at 500 Kbps). And when the pushing server sends pushing information for indicating the pushing end to push the current with a lower bit rate than the current pushing rate to the pushing end, the pushing end needs to push the current with a lower bit rate than the current pushing rate after receiving the pushing information (for example, the pushing rate is reduced from 500Kbps to 250Kbps to the pushing server).

It can be appreciated that, due to the phenomenon of being changed by the network environment, the network situation where the push end is located at different moments may occur in a state that is smoother or more congested than the previous moment. Therefore, in the embodiment of the application, after receiving the audio and video streams sent by the push end at intervals, the push server needs to generate corresponding push information according to the network condition under the current received audio and video, so that the frequency and time consumption of code stream switching caused by the possibility of blocking the audio and video streams in an oscillation network are reduced, the low delay of streaming media transmission is further ensured, and the problem of blocking the audio and video caused by the oscillation of the network environment where the push end equipment is located in the live broadcast process of a user is avoided.

In one mode, in the process of generating the push information according to the network operation state, the push server can detect at least one of a packet loss rate parameter, a transmission rate parameter and an interval jitter parameter in the current network state to determine the current network operation state after receiving the audio/video stream with the first code rate. For example, the higher the packet loss rate, the more congested the network operation state is represented, while the faster the transmission rate and the smaller the interval jitter, the more unobstructed the network operation state is represented.

S103, the plug flow information is sent to a plug flow end.

S104, receiving the audio and video stream pushed by the pushing end according to the second code rate determined by the pushing information.

Furthermore, the push information has a push code rate (i.e. a second code rate) for informing the push end of pushing the audio and video stream next time. And the push server can receive the audio/video stream pushed according to the second code rate indicated by the push information after sending the push information to the push end. It is understood that the second code rate may be an increased or decreased code rate or a constant code rate as compared to the first code rate.

In the application, after the push server receives the audio and video stream pushed by the push terminal with the first code rate, the network operation state of the received audio and video stream can be detected, push information for indicating the push code rate of the next push audio and video stream of the push terminal is generated according to the network operation state, and then the push information is sent to the push terminal, and then the push terminal is received to push the audio and video stream with the second code rate determined according to the push information. By applying the technical scheme, the push server can pointedly generate push information for informing the push end of pushing the push code rate matched with the network operation condition next time according to the network operation condition of the audio and video stream pushed by the push end currently, so that the frequency and time consumption of code stream switching caused by the fact that the audio and video stream is blocked in an oscillation network can be reduced, the low delay of streaming media transmission is further ensured, and the problem that the audio and video is blocked due to the oscillation of the network environment where the push end equipment is located in the live broadcast process of a user is avoided.

Alternatively, in another embodiment based on the above method of the present application, in S102 (detecting the network operation state of receiving the audio/video stream, generating the push information according to the network operation state), the following steps may be implemented:

if the network running state is determined to correspond to the network light load, generating pushing information for indicating a pushing end to push at a higher rate than the last pushing; or alternatively, the first and second heat exchangers may be,

if the network running state is determined to correspond to the network normal, generating pushing information for indicating a pushing end to continue pushing with the latest pushing code rate; or alternatively, the first and second heat exchangers may be,

and if the network operation state is determined to correspond to network congestion, generating pushing information for indicating a pushing end to push at a rate not higher than the last pushing rate.

Taking the latest push code rate of the push end as an example, when the push server receives the audio and video stream pushed by the push end with the 500Kbps code rate, the push server can determine the current network running state according to at least one of the packet loss rate parameter, the transmission rate parameter and the interval jitter parameter in the current network state. When it is determined that the network operation state corresponds to the network light load, push information for instructing the push terminal to push at a code rate higher than 500Kbps code rate (for example, 1000Kbps, 2000Kbps, etc.) is required to be generated. When the network operation state is determined to correspond to the network normal, generating push information for indicating the push terminal to continue pushing at the 500Kbps code rate is needed. When it is determined that the network operation state corresponds to network congestion, it is necessary to generate push information for instructing the push terminal to push at a code rate lower than 500Kbps code rate (for example, 150Kbps, 250Kbps, etc.).

In one mode, in the push information generated by the push server, a push code rate value or a push gear level can be explicitly fed back, so that the push end takes the code rate value or the code rate value corresponding to the push gear level as the next push code rate.

In another way, a high-low relation with the last push code rate can be fed back, so that the push terminal can select the next push code rate according to the high-low relation. For example, taking the last push code rate of the push end as an example, when the push server determines that the network operation state corresponds to the network light load, a push message for indicating the push end to push at a higher rate than the last push code rate can be generated, and after receiving the push message, the push end can select a code rate (for example, 1000Kbps, 2000Kbps, etc.) higher than the 500Kbps code rate by itself to push next time.

Optionally, the generating, for indicating the push end to push at a higher rate than the last push rate, the push information that is generated by the above-mentioned network operation state corresponds to a network light load may further include:

In one mode, when determining that the network operation state corresponds to the network light load, the present application needs to generate push information for indicating the push end to push at a higher rate than the last push. And when the number of code rates higher than the last time of the plug-in code rate is a plurality of, selecting the code rate closest to the last time of the plug-in code rate from a plurality of preset code rates to generate the plug-in information.

For example, when the latest push rate is 500Kbps, the rate higher than 500Kbps is plural, including 1000Kbps,2000Kbps,4000Kbps,6000Kbps, etc. In the embodiment of the present application, the code rate adjacent to the code rate needs to be selected from the code rates above 500Kbps to be used as the code rate for indicating the pushing end to perform the pushing next time (i.e. 1000 Kbps). And generating corresponding plug flow information according to the code rate and sending the plug flow information to a plug flow end.

In another mode, a plurality of different gear levels can be generated in advance, and corresponding code rates are set under each gear level, so that when the push server generates push information, the code rate corresponding to the gear level most adjacent to the last push code rate can be selected to generate the push information according to the gear level where the last push code rate is located.

Specifically, as shown in table 1 below, a plurality of gear steps are included:

for example, when the last push rate is 500Kbps (i.e. corresponds to the 3 rd gear level), when the push server determines that the current network condition is a light load, the rate of the 4 th gear level (i.e. 1000 Kbps) adjacent to the 3 rd gear level may be selected as the rate (i.e. 1000 Kbps) indicating that the push terminal performs push next time, and then corresponding push information is generated according to the rate and sent to the push terminal. Therefore, the aim of gradually improving the code rate of the received push audio and video can be achieved, and the problem that network stability is affected due to the fact that the code rate spans are high when adjacent audio and video streams with different code rates are received twice is avoided.

Optionally, after generating the push information for indicating the push end to push at a higher rate than the last push rate, the above mentioned network operation state corresponds to a network light load, the method may further include:

and if the latest push code rate reaches the preset threshold, generating push information for indicating the push terminal to push at a code rate higher than and adjacent to the latest push code rate only when the audio/video stream of which the network operation state corresponds to the network light load is detected to be continuously received for multiple times.

In one mode, after determining that the code rate of the last push of the push end reaches a higher threshold, if the push server detects that the audio and video stream pushed by the push end is received for a plurality of times and the detection result is that the network running state corresponds to the network light load, the current network running state can be determined to be relatively stable.

In one mode, in order to achieve the purpose of keeping the push rate relatively stable when the push end is in a stable operation network state, in the process of receiving an audio and video stream by a subsequent push server, the embodiment of the application can generate push information for indicating the push end to push at a code rate higher than and adjacent to the last push code rate only when the audio and video stream corresponding to the network light load in the network operation state is received for a plurality of times. Therefore, the purpose that the pushing end is positioned in a stable operation network state and the code rate of the pushing end is not required to be changed frequently is ensured.

The number of times is not limited in this application, and may be, for example, 2 times, 5 times, or the like.

For example, taking a preset threshold value of 1000Kbps and the number of times of 2 as an example, when the push server detects that the audio and video stream pushed by the push end reaches 1000Kbps, when the audio and video stream is received twice subsequently and the network running states of the server correspond to the network light load, the network running state of the audio and video stream is continuously detected for the third time, and when the audio and video stream is received for the third time and the network running states of the server correspond to the network light load, push information for indicating the push end to push at a code rate higher than and adjacent to the last push code rate is generated. And then the push information is sent to the push end.

Optionally, if it is determined that the network operation state corresponds to network congestion, generating the push information for indicating the push end to push at a rate not higher than the last push rate may further include:

if the network running state is determined to correspond to network congestion, generating pushing information for indicating a pushing end to push with an initial pushing code rate, wherein the initial pushing code rate is a code rate corresponding to the pushing end to push the audio and video stream to the pushing server for the first time.

In one mode, when determining that the network operation state corresponds to network congestion, the present application needs to generate push information for indicating a push end to push at a rate not higher than the last push rate. When the number of code rates not higher than the last push code rate is multiple, the embodiment of the application needs to select the code rate (i.e. the initial push code rate) corresponding to the push terminal pushing the audio and video stream to the push server for the first time to generate push information.

In one mode, the initial push code rate may be the lowest code rate that the push terminal can reach in its current network condition, or may be the next lowest code rate that the push terminal can reach in its current network condition, etc.

Optionally, the above-mentioned push information for indicating the push end to push at the initial push code rate may further include:

and if the network operation state is determined to correspond to network congestion, generating pushing information for indicating a pushing end to push at the lowest code rate.

In one mode, taking the initial push code rate as the lowest code rate (for example, 500 Kbps) as an example, when the push server determines that the network operation state corresponds to the network light load for multiple times, and generates push information for indicating the push end to push at a higher code rate than the last push code rate for multiple times, and sends the push information to the push end, the push end gradually pushes at a higher code rate than the last time until the 120000Kbps code rate is reached. When the push server receives the audio/video stream pushed by the push end at 120000Kbps code rate and detects that the current running state corresponds to network congestion, the problem that the push audio/video stream is not matched with the network condition because the optimal code rate corresponding to the next second of network running condition and the current calculated optimal code rate are changed due to the network problem is solved. According to the method and the device, the pushing server can directly generate the pushing information indicating the pushing end to push the stream with the lowest code rate (namely 1000 Kbps) and send the pushing information to the pushing end, so that the follow-up receiving of the audio and video stream pushed by the pushing end to push the stream with the 500Kbps code rate is realized. Thereby ensuring the stability of audio and video transmission.

Optionally, in another embodiment based on the above method of the present application, after S102 (detecting the network operation state of receiving the audio/video stream, generating the push information according to the network operation state), the following steps may be implemented:

if the condition that the same network operation state is continuously received for multiple times is detected, increasing or reducing the time interval for sending the push stream information;

and sending the push information to a push end according to the time interval.

In one mode, if the push server receives the audio and video streams corresponding to the same network operation state for a plurality of times, the current network condition can be determined to be stable. Therefore, in order to quickly receive the audio/video stream with the ideal code rate, in the embodiment of the application, the time interval for sending the push information to the push end can be correspondingly adjusted by the push server. So that the push server receives the radio video stream rate at a frequency that matches the current network operating state.

In one manner, for example, after detecting that the network operation state corresponding to the network light load is received continuously multiple times, the time interval for transmitting the push stream information may be shortened. Specifically, after receiving the audio and video stream corresponding to the network light load for multiple times, the push server can determine that the current network condition is better, so that the push server can push the audio and video stream with higher code rate more quickly by sending the push information at a shorter time interval, and further, the aim of enabling the audio and video stream pushed by the push server to quickly reach a better picture in the network environment under the condition of ensuring the better network condition is fulfilled.

Further, taking the original time interval of 1s and the initial push rate of 500Kbps as an example, if the push server sends 3 push messages to the push end to respectively inform the push end to push the audio and video stream with higher rate than the previous time (for example, respectively inform the next push rate of 1000Kbps,2000Kbps and 4000 Kbps), the time from the initial push rate (500 Kbps) to receiving the audio and video stream with 4000Kbps is 3 seconds. And if the time interval for transmitting the push information is shortened (for example, 0.5 s), the time from the initial push rate (500 Kbps) to receiving the audio/video stream with 4000Kbps is only 1.5 seconds. Therefore, the aim of rapidly receiving the audio and video stream with the ideal code rate under the current network condition is fulfilled.

In another manner, for example, the time interval for transmitting push messages may be increased after detecting that a network operational state corresponding to network congestion is received a number of times in succession. Specifically, the push server can determine that the current network condition is poor after receiving the audio and video stream corresponding to the operation state of the congestion network for a plurality of times, so that frequent sending of push information also causes resource receiving and transmitting of the server to be wasted. In view of this, the time interval for transmitting push information may be increased (e.g., from every 1s to every 3 s) in the embodiments of the present application. Therefore, the frequency of sending the push flow information is reduced as much as possible in the time period of unstable network operation condition of the push flow server, and the purpose of saving operation resources is achieved.

and determining the network operation state of the received audio and video stream based on at least one of the packet loss rate parameter, the transmission rate parameter and the interval jitter parameter.

In one manner, the embodiment of the application can determine the network operation state of receiving the radio video stream by the server by detecting one of the packet loss rate parameter, the transmission rate parameter and the interval jitter parameter of the current network.

It will be appreciated that, for example, the higher the packet loss rate, the more congested the network operation state is represented, while the faster the transmission rate and the smaller the interval jitter, the more unobstructed the network operation state is represented. In another manner, the network operation state of the received audio/video stream may be determined according to other parameters. The present application is not limited in this regard.

In an embodiment, the present application further provides a method for pushing an audio/video stream. Fig. 2 schematically shows a flow diagram of a method for pushing an audio/video stream according to an embodiment of the present application. As shown in fig. 2, the method is applied to a push end, and includes:

S201, pushing the audio and video stream to a push server at a first code rate.

The first code rate is not specifically limited, for example, the audio and video stream may be pushed to the push server with a fixed code rate, or the audio and video stream may be pushed to the push server with a code rate that is the most matched in the network environment where the push terminal is located.

It should be noted that, the push server mentioned in the present application may be an edge server under a content delivery network (content delivery network, CDN) network, or may be a regional server, or may be a central server. It should be noted that, when the push server is an edge server, the push end may be a client end (e.g., a live end) used by a user. When the push server is a regional server or a central server, the push end may be an edge server in communication with the regional server or a regional server in communication with the central server. It will be appreciated that the edge server communicates with the clients used by the user.

S202, receiving push information generated by a push server based on the audio and video stream.

After receiving the audio/video stream with the first code rate pushed by the pushing end, the pushing server in the embodiment of the application can dynamically generate the pushing information for indicating the pushing code rate of the audio/video stream pushed by the pushing end next time in real time according to the current network running state of the pushing server, and send the pushing information to the pushing end so as to indicate the pushing code rate of the audio/video stream pushed by the pushing end next time.

And S203, pushing the audio and video stream to the push server at a second code rate represented by the push information.

In one mode, the second code rate may be the same as the first code rate or may be different. After receiving the push information, the push terminal can directly analyze the information to obtain the push code rate carried in the information for indicating the next push audio/video stream.

Specifically, when the push server sends push information for instructing the push end to push at a higher rate than the last push rate to the push end, the push end needs to push at a higher rate than the last push rate (e.g., push at a second rate from a first rate of 500Kbps to 1000Kbps to the push server) after receiving the push information. And when the push server sends push information for indicating the push end to continue pushing with the last push code rate to the push end, the push end needs to continue pushing with the last push code rate after receiving the push information (for example, maintaining the push to the push server at 500 Kbps). And when the pushing server sends pushing information for indicating the pushing end to push the current with a lower bit rate than the current pushing rate to the pushing end, the pushing end needs to push the current with a lower bit rate than the current pushing rate after receiving the pushing information (for example, pushing the current to the pushing server from a first bit rate of 500Kbps to a second bit rate of 250 Kbps).

Optionally, in another embodiment based on the above method of the present application, in S201 (pushing the audio and video stream to the push server with the first code rate), the following steps may be implemented:

And if the fact that the audio and video streams are pushed to the push server for the first time is determined, pushing the audio and video streams to the push server with the lowest code rate.

The minimum code rate is not specifically limited in the present application, and may be, for example, the minimum code rate that can be achieved by the push end in the current network environment. The code rate corresponding to the lowest gear level in the preset plurality of different gear levels can also be obtained.

Further, in the present application, as illustrated in fig. 3, it is first required that the push end pushes the audio and video stream to the push server with a first code rate, so that the push server detects a network running state of receiving the audio and video stream, generates push information for indicating a push code rate of the audio and video stream that is pushed next by the push end according to the network running state, sends the push information to the push end, and finally, the push end pushes the audio and video stream to the push server with a second code rate represented by the push information.

In another embodiment of the present application, as shown in fig. 4, the present application further provides an electronic device for pushing an audio/video stream. The method is applied to a push server, and comprises a first receiving module, a first generating module, a sending module and a second receiving module, wherein the first receiving module comprises:

the first receiving module 301 is configured to receive an audio/video stream pushed by the pushing end at a first code rate;

the first generating module 302 is configured to detect a network operation state of receiving the audio and video stream, and generate push information according to the network operation state, where the push information is used to indicate a push code rate of pushing the audio and video stream by the push end next time;

a sending module 303, configured to send the push information to the push end;

the second receiving module 304 is configured to receive the audio/video stream pushed by the pushing end at the second code rate determined according to the pushing information.

In the application, after the push server receives the audio and video stream pushed by the push terminal with the first code rate, the network operation state of the received audio and video stream can be detected, push information for indicating the push code rate of the next push audio and video stream of the push terminal is generated according to the network operation state, and then the push information is sent to the push terminal, and then the push terminal is received to push the audio and video stream with the second code rate determined according to the push information. By applying the technical scheme, the push server can receive the network operation condition of the audio and video stream pushed by the push terminal according to the current self, and the push server can generate the push information of the push code rate for informing the push terminal of pushing the audio and video stream matched with the network operation condition next time in a targeted manner, so that the frequency and time consumption of code stream switching caused by the fact that the audio and video stream is blocked in an oscillation network can be reduced, the low delay of streaming media transmission is further ensured, and the problem that the audio and video is blocked due to the oscillation of the network environment where the push terminal equipment is located in the live broadcast process of a user is avoided.

In another embodiment of the present application, the first generating module 302 is configured to:

and the initial push code rate is the lowest code rate of pushing the audio and video stream to the push server by the push terminal.

In another embodiment of the present application, the sending module 303 is configured to:

In another embodiment of the present application, as shown in fig. 5, the present application further provides an electronic device for pushing an audio/video stream. Is applied to plug flow end, includes:

a first pushing module 305 configured to push the audio and video stream to the push server at a first code rate;

a second generating module 306 configured to receive push information generated by the push server based on the audio/video stream;

and a second pushing module 307 configured to push the audio/video stream to the push server at a second code rate characterized by the push information.

In the application, after the push server receives the audio and video stream pushed by the push terminal with the first code rate, the network operation state of the received audio and video stream can be detected, push information for indicating the push code rate of the next push audio and video stream of the push terminal is generated according to the network operation state, and then the push information is sent to the push terminal, and then the push terminal is received to push the audio and video stream with the second code rate determined according to the push information. By applying the technical scheme, the push server can generate the push information informing the push terminal to push the push code rate matched with the network operation condition next time according to the network operation condition under the audio and video pushed by the current receiving push terminal, so that the aim of informing the push terminal to adjust the push of the audio and video streams with different code rates in real time is achieved. And further, the problem that a user generally causes video blocking due to the vibration of the network environment where the equipment is located in the live broadcast process is avoided.

In another embodiment of the present application, the first pushing module 305 is configured to:

Fig. 6 is a block diagram of a logic structure of an electronic device, according to an example embodiment. For example, electronic device 400 may be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, medical device, exercise device, personal digital assistant, or the like.

Referring to fig. 6, electronic device 400 may include one or more of the following components: a processor 401 and a memory 402.

Processor 401 may include one or more processing cores such as a 4-core processor, an 8-core processor, etc. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 401 may also include a main processor, which is a processor for processing data in an awake state, also called a CPU (Central Processing Unit ), and a coprocessor; a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 401 may integrate a GPU (Graphics Processing Unit, image processor) for rendering and drawing of content required to be displayed by the display screen. In some embodiments, the processor 401 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

Memory 402 may include one or more computer-readable storage media, which may be non-transitory. Memory 402 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 402 is used to store at least one instruction for execution by processor 401 to implement the interactive special effects calibration method provided by the method embodiments herein.

In some embodiments, the electronic device 400 may further optionally include: a peripheral interface 403 and at least one peripheral. The processor 401, memory 402, and peripheral interface 403 may be connected by a bus or signal line. The individual peripheral devices may be connected to the peripheral device interface 403 via buses, signal lines or a circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 404, a touch display 405, a camera 406, audio circuitry 407, a positioning component 408, and a power supply 409.

Peripheral interface 403 may be used to connect at least one Input/Output (I/O) related peripheral to processor 401 and memory 402. In some embodiments, processor 401, memory 402, and peripheral interface 403 are integrated on the same chip or circuit board; in some other embodiments, either or both of the processor 401, memory 402, and peripheral interface 403 may be implemented on separate chips or circuit boards, which is not limited in this embodiment.

The Radio Frequency circuit 404 is configured to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuitry 404 communicates with a communication network and other communication devices via electromagnetic signals. The radio frequency circuit 404 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 404 includes: antenna systems, RF transceivers, one or more amplifiers, tuners, oscillators, digital signal processors, codec chipsets, subscriber identity module cards, and so forth. The radio frequency circuitry 404 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and/or WiFi (Wireless Fidelity ) networks. In some embodiments, the radio frequency circuitry 404 may also include NFC (Near Field Communication ) related circuitry, which is not limited in this application.

The display screen 405 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 405 is a touch display screen, the display screen 405 also has the ability to collect touch signals at or above the surface of the display screen 405. The touch signal may be input as a control signal to the processor 401 for processing. At this time, the display screen 405 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display screen 405 may be one, providing a front panel of the electronic device 400; in other embodiments, the display screen 405 may be at least two, and disposed on different surfaces of the electronic device 400 or in a folded design; in still other embodiments, the display 405 may be a flexible display disposed on a curved surface or a folded surface of the electronic device 400. Even more, the display screen 405 may be arranged in an irregular pattern that is not rectangular, i.e. a shaped screen. The display 405 may be made of LCD (Liquid Crystal Display ), OLED (Organic Light-Emitting Diode) or other materials.

The camera assembly 406 is used to capture images or video. Optionally, camera assembly 406 includes a front camera and a rear camera. Typically, the front camera is disposed on the front panel of the terminal and the rear camera is disposed on the rear surface of the terminal. In some embodiments, the at least two rear cameras are any one of a main camera, a depth camera, a wide-angle camera and a tele camera, so as to realize that the main camera and the depth camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize a panoramic shooting and Virtual Reality (VR) shooting function or other fusion shooting functions. In some embodiments, camera assembly 406 may also include a flash. The flash lamp can be a single-color temperature flash lamp or a double-color temperature flash lamp. The dual-color temperature flash lamp refers to a combination of a warm light flash lamp and a cold light flash lamp, and can be used for light compensation under different color temperatures.

The audio circuit 407 may include a microphone and a speaker. The microphone is used for collecting sound waves of users and environments, converting the sound waves into electric signals, and inputting the electric signals to the processor 401 for processing, or inputting the electric signals to the radio frequency circuit 404 for realizing voice communication. For purposes of stereo acquisition or noise reduction, the microphone may be multiple and separately disposed at different locations of the electronic device 400. The microphone may also be an array microphone or an omni-directional pickup microphone. The speaker is used to convert electrical signals from the processor 401 or the radio frequency circuit 404 into sound waves. The speaker may be a conventional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, not only the electric signal can be converted into a sound wave audible to humans, but also the electric signal can be converted into a sound wave inaudible to humans for ranging and other purposes. In some embodiments, audio circuit 407 may also include a headphone jack.

The location component 408 is used to locate the current geographic location of the electronic device 400 to enable navigation or LBS (Location Based Service, location-based services). The positioning component 408 may be a positioning component based on the united states GPS (Global Positioning System ), the beidou system of china, the grainer system of russia, or the galileo system of the european union.

The power supply 409 is used to power the various components in the electronic device 400. The power supply 409 may be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When power supply 409 comprises a rechargeable battery, the rechargeable battery may support wired or wireless charging. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, the electronic device 400 further includes one or more sensors 410. The one or more sensors 410 include, but are not limited to: acceleration sensor 411, gyroscope sensor 412, pressure sensor 413, fingerprint sensor 414, optical sensor 415, and proximity sensor 416.

The acceleration sensor 411 may detect the magnitudes of accelerations on three coordinate axes of the coordinate system established with the electronic device 400. For example, the acceleration sensor 411 may be used to detect components of gravitational acceleration on three coordinate axes. The processor 401 may control the touch display screen 405 to display a user interface in a lateral view or a longitudinal view according to the gravitational acceleration signal acquired by the acceleration sensor 411. The acceleration sensor 411 may also be used for the acquisition of motion data of a game or a user.

The gyro sensor 412 may detect a body direction and a rotation angle of the electronic device 400, and the gyro sensor 412 may collect a 3D motion of the user on the electronic device 400 in cooperation with the acceleration sensor 411. The processor 401 may implement the following functions according to the data collected by the gyro sensor 412: motion sensing (e.g., changing UI according to a tilting operation by a user), image stabilization at shooting, game control, and inertial navigation.

The pressure sensor 413 may be disposed at a side frame of the electronic device 400 and/or at an underlying layer of the touch screen 405. When the pressure sensor 413 is disposed on a side frame of the electronic device 400, a grip signal of the user on the electronic device 400 may be detected, and the processor 401 performs a left-right hand recognition or a shortcut operation according to the grip signal collected by the pressure sensor 413. When the pressure sensor 413 is disposed at the lower layer of the touch display screen 405, the processor 401 controls the operability control on the UI interface according to the pressure operation of the user on the touch display screen 405. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

The fingerprint sensor 414 is used to collect a fingerprint of the user, and the processor 401 identifies the identity of the user based on the fingerprint collected by the fingerprint sensor 414, or the fingerprint sensor 414 identifies the identity of the user based on the collected fingerprint. Upon recognizing that the user's identity is a trusted identity, the user is authorized by the processor 401 to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying for and changing settings, etc. The fingerprint sensor 414 may be provided on the front, back, or side of the electronic device 400. When a physical key or vendor Logo is provided on the electronic device 400, the fingerprint sensor 414 may be integrated with the physical key or vendor Logo.

The optical sensor 415 is used to collect the ambient light intensity. In one embodiment, the processor 401 may control the display brightness of the touch display screen 405 according to the ambient light intensity collected by the optical sensor 415. Specifically, when the intensity of the ambient light is high, the display brightness of the touch display screen 405 is turned up; when the ambient light intensity is low, the display brightness of the touch display screen 405 is turned down. In another embodiment, the processor 401 may also dynamically adjust the shooting parameters of the camera assembly 406 according to the ambient light intensity collected by the optical sensor 415.

A proximity sensor 416, also referred to as a distance sensor, is typically provided on the front panel of the electronic device 400. The proximity sensor 416 is used to collect distance between the user and the front of the electronic device 400. In one embodiment, when the proximity sensor 416 detects a gradual decrease in the distance between the user and the front of the electronic device 400, the processor 401 controls the touch display 405 to switch from the bright screen state to the off screen state; when the proximity sensor 416 detects that the distance between the user and the front surface of the electronic device 400 gradually increases, the processor 401 controls the touch display screen 405 to switch from the off-screen state to the on-screen state.

Those skilled in the art will appreciate that the structure shown in fig. 4 is not limiting of the electronic device 400 and may include more or fewer components than shown, or may combine certain components, or may employ a different arrangement of components.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium, such as memory 404, including instructions executable by processor 420 of electronic device 400 to perform the above-described method of audio-video stream-based push, the method comprising: receiving an audio and video stream pushed by a pushing end at a first code rate; detecting a network running state of receiving the audio and video stream, and generating push information according to the network running state, wherein the push information is used for indicating a push code rate of pushing the audio and video stream next time by the push terminal; the plug flow information is sent to the plug flow end; and receiving the audio and video stream pushed by the pushing end according to the second code rate determined by the pushing information. Optionally, the above instructions may also be executed by the processor 420 of the electronic device 400 to perform the other steps involved in the above-described exemplary embodiments. For example, the non-transitory computer readable storage medium may be ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

In an exemplary embodiment, there is also provided an application/computer program product comprising one or more instructions executable by the processor 420 of the electronic device 400 to perform the above-described method of push based on an audio video stream, the method comprising: receiving an audio and video stream pushed by a pushing end at a first code rate; detecting a network running state of receiving the audio and video stream, and generating push information according to the network running state, wherein the push information is used for indicating a push code rate of pushing the audio and video stream next time by the push terminal; the plug flow information is sent to the plug flow end; and receiving the audio and video stream pushed by the pushing end according to the second code rate determined by the pushing information. Optionally, the above instructions may also be executed by the processor 420 of the electronic device 400 to perform the other steps involved in the above-described exemplary embodiments.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. The method for pushing the audio and video streams is characterized by being applied to a push server and comprising the following steps:

the plug flow information is sent to the plug flow end;

2. The method of claim 1, wherein the generating plug flow information based on the network operating state comprises:

3. The method of claim 2, wherein if the network operation state is determined to correspond to a network light load, generating the push information for indicating the push terminal to push at a higher rate than the last push rate comprises:

4. The method of claim 2 or 3, further comprising, after the generating the push information for instructing the push end to push at a higher rate than the last push:

5. The method of claim 2, wherein if the network operation state is determined to correspond to network congestion, generating push information for indicating the push terminal to push at a rate not higher than a last push rate comprises:

6. The method of claim 5, wherein the initial push rate is a lowest rate at which the push end pushes the audio-video stream to the push server.

7. The method of claim 1, further comprising, after said generating push information from said network operational state:

8. The method of claim 1, wherein detecting the network operational status of receiving the audio-video stream comprises:

9. The method for pushing the audio and video streams is characterized by being applied to a pushing end and comprising the following steps:

pushing the audio and video stream to a push server at a first code rate;

receiving push information generated by the push server based on the network running state of the audio and video stream;

10. The method of claim 9, wherein pushing the audio-video stream to the push server at the first code rate comprises:

11. An electronic device for pushing an audio and video stream, which is applied to a push server, comprises:

The sending module is configured to send the push information to the push end;

12. An electronic device for pushing an audio and video stream, which is characterized by being applied to a pushing end, comprising:

13. An electronic device, comprising:

a memory for storing executable instructions; the method comprises the steps of,

a processor for displaying with the memory to execute the executable instructions to perform the operations of the method of audio video streaming of any of claims 1-8 or 9-10.

14. A computer readable storage medium storing computer readable instructions that when executed perform the operations of the method of audio video streaming of any of claims 1-8 or 9-10.

15. A computer program product comprising a computer program, characterized in that the computer program is executed by a processor to implement the method of any one of claims 1-8 or 9-10.