CN113271496B - Video smooth playing method and system in network live broadcast and readable storage medium - Google Patents
Video smooth playing method and system in network live broadcast and readable storage medium Download PDFInfo
- Publication number
- CN113271496B CN113271496B CN202110799460.5A CN202110799460A CN113271496B CN 113271496 B CN113271496 B CN 113271496B CN 202110799460 A CN202110799460 A CN 202110799460A CN 113271496 B CN113271496 B CN 113271496B
- Authority
- CN
- China
- Prior art keywords
- playing
- video
- frames
- video frames
- smooth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
Abstract
The invention discloses a method, a system and a readable storage medium for smoothly playing videos in live webcasting, wherein the method comprises the following steps: the method comprises the steps of establishing a smooth window containing a preset number of virtual frames in a cache queue, calculating and playing the smooth time of the video frames in the smooth window according to the product of the number of the virtual frames and the number of the video frames played in the cache queue, when the video frames are stacked in the cache queue, putting the stacked video frames into the smooth window, dividing the number of the frames in the smooth window by the smooth time to obtain the playing duration of the stacked video frames, and playing the video frames according to the playing duration, so that the effect of reasonably and effectively processing uneven video streams is achieved under the condition that the video frames are not discarded and the time delay is not increased, the smooth playing effect of the video is ensured, and the problem of poor experience of watching users caused by network blockage is solved.
Description
Technical Field
The invention relates to the field of network live broadcast, in particular to a method and a system for smoothly playing videos in network live broadcast and a readable storage medium.
Background
With the development of internet network technology, the network live broadcast industry is rapidly developing and growing, and now plays a very important role in daily life of people, and particularly plays a prominent role in playing games, entertainment, shopping, activity conferences and other links, so that the requirements of people on the real-time performance and the fluency of live broadcast are higher and higher. However, in a wide area network, network instability sometimes occurs, when the network is abnormal, a video frame sent from a live broadcast end cannot be transmitted to a play end in real time, the picture of the play end is stagnated, when the network is recovered, the play end has a situation of video frame accumulation, in order to ensure a subsequent live broadcast effect, the accumulated video frame picture is directly skipped over after the network is recovered, and a viewer may miss important information.
In order to ensure the smoothness of watching live video, a corresponding method is needed to deal with the smoothness. The current commonly used mode for ensuring the fluency of live video is to reduce the unstable effect caused by network jitter through delay play and increase of the buffer number, but the requirement on the delay is higher in scenes, for example, when live activities needing real-time interaction occur, the effect of the live activities can be directly influenced by overhigh delay.
Therefore, how to provide a method for smoothly playing a video stream for network transmission, which reasonably and effectively processes an uneven video stream, ensures the effect of smoothly playing the video without increasing the delay as much as possible, and improves the experience of watching the video by a user in live network broadcast is a technical problem to be solved by technical personnel in the field.
Disclosure of Invention
In view of the foregoing problems, an object of the present invention is to provide a method, a system, and a readable storage medium for smoothly playing a video in live webcast, which ensure a smooth playing effect of the video without increasing a delay as much as possible, and improve a user experience of watching the video in live webcast.
The invention provides a video smooth playing method in network live broadcast, which comprises the following steps:
s102, creating a smooth window containing a preset number of virtual frames in a buffer queue;
s104, acquiring the average playing time of the video frames;
s106, calculating the smooth time for playing the video frame in the smooth window according to the product of the average time and the number of the virtual frames;
s108, when video frames are accumulated in the buffer queue, the accumulated video frames are placed in a smooth window;
s110, dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time length of the accumulated video frames;
and S112, playing the video frame according to the playing time length, and returning to the step S108.
In this scheme, the obtaining of the average time for playing the video frame specifically includes:
and acquiring the average time of sending the video frames by the live broadcast end.
In this scheme, the obtaining of the average time for playing the video frame specifically includes:
and calculating the average time of video frame playing according to the received playing time of the picture group and the total amount of the video frames in the picture group.
In this scheme, when video frames are accumulated in the buffer queue, the step of placing the accumulated video frames into the smooth window specifically includes:
when the number of the video frames in the buffer queue is 0, receiving video data;
when the number of the video frames in the buffer queue is 1, directly playing the video frames;
and when the number of the video frames in the buffer queue is more than 1, putting the piled video frames into a smooth window.
In this embodiment, step S10 specifically includes: dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time length of the video frame positioned in the first order in the piled video frames;
the step S12 specifically includes: and playing the video frame positioned in the first sequence according to the playing time length, and returning to the step S108.
The second aspect of the present invention further provides a system for smoothly playing videos in live webcasts, including a memory and a processor, where the memory includes a program of a method for smoothly playing videos in live webcasts, and when executed by the processor, the program of the method for smoothly playing videos in live webcasts realizes the following steps:
s102, creating a smooth window containing a preset number of virtual frames in a buffer queue;
s104, acquiring the average playing time of the video frames;
s106, calculating the smooth time for playing the video frame in the smooth window according to the product of the average time and the number of the virtual frames;
s108, when video frames are accumulated in the buffer queue, the accumulated video frames are placed in a smooth window;
s110, dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time length of the accumulated video frames;
and S112, playing the video frame according to the playing time length, and returning to the step S108.
In this scheme, the obtaining of the average time for playing the video frame specifically includes:
and calculating the average time of video frame playing according to the received playing time of the picture group and the total amount of the video frames in the picture group.
In this scheme, the obtaining of the average time for playing the video frame specifically includes: when the video frames are accumulated in the buffer queue, the step of placing the accumulated video frames into the smoothing window specifically includes:
when the number of the video frames in the buffer queue is 0, receiving video data;
when the number of the video frames in the buffer queue is 1, directly playing the video frames;
and when the number of the video frames in the buffer queue is more than 1, putting the piled video frames into a smooth window.
In this embodiment, step S10 specifically includes: dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time length of the video frame positioned in the first order in the piled video frames;
the step S12 specifically includes: and playing the video frame positioned in the first sequence according to the playing time length, and returning to the step S108.
A third aspect of the present invention provides a computer-readable storage medium, where the computer-readable storage medium includes a program of a video smooth playing method in live webcasting of a machine, and when the program of the video smooth playing method in live webcasting is executed by a processor, the method implements the steps of the video smooth playing method in live webcasting as described in any one of the above.
The invention discloses a method, a system and a readable storage medium for smoothly playing videos in live webcasting, wherein the method comprises the following steps: the method comprises the steps of establishing a smooth window containing a preset number of virtual frames in a cache queue, calculating and playing the smooth time of the video frames in the smooth window according to the product of the number of the virtual frames and the number of the video frames played in the cache queue, when the video frames are stacked in the cache queue, putting the stacked video frames into the smooth window, dividing the number of the frames in the smooth window by the smooth time to obtain the playing duration of the stacked video frames, and playing the video frames according to the playing duration, so that the effect of reasonably and effectively processing uneven video streams is achieved under the condition that the video frames are not discarded and the time delay is not increased, the smooth playing effect of the video is ensured, and the problem of poor experience of watching users caused by network blockage is solved.
Drawings
FIG. 1 is a flow chart illustrating a video smooth playing method in live webcasting according to the present application;
fig. 2 shows a block diagram of a video smooth-casting system in webcasting according to the present invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments and features of the embodiments of the present application may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited by the specific embodiments disclosed below.
Fig. 1 shows a flowchart of a video smooth playing method in live webcasting according to the present application.
As shown in fig. 1, the present application discloses a method for smoothly playing a video in live webcasting, which includes the following steps:
s102, creating a smooth window containing a preset number of virtual frames in a buffer queue;
s104, acquiring the average playing time of the video frames;
s106, calculating the smooth time for playing the video frame in the smooth window according to the product of the average time and the number of the virtual frames;
s108, when video frames are accumulated in the buffer queue, the accumulated video frames are placed in a smooth window;
s110, dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time length of the accumulated video frames;
and S112, playing the video frame according to the playing time length, and returning to the step S108.
It should be noted that, after the live broadcast end receives the video data, the buffering of the video frames is performed in the buffer queue, and the smooth window is set in the buffer queue. The average time of playing video frames, i.e. the reciprocal of the video frame rate, is the dwell time of each video frame picture. The smoothing time refers to the total time of playing the video frames (including the virtual frames) in the smoothing window, that is, the video frames are played in the smoothing time no matter how many frames are in the smoothing window, so the smoothing time is the product of the average time and the number of the virtual frames. When accumulation occurs in the buffer queue, which indicates that a network is unstable before, the video frame which should be received originally and the video frame at the current moment are received into the buffer queue at the same time. And putting the piled video frames into a smooth window, wherein the frames in the smooth window are the sum of the video frames and the virtual frames. And dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time of each frame of the accumulated video frames, namely, the accumulated video frames are subjected to smoothing transition by using the time of the virtual frames. Compared with the method for directly skipping the accumulated video frames or improving the delay playing time in order to ensure the live broadcasting effect in the prior art, the method provided by the invention can achieve the effects of reasonably and effectively processing the uneven video stream and ensuring the smooth playing of the video under the condition of not discarding the video frames and not improving the delay, and solves the problem of poor user viewing experience caused by network blockage.
The following are illustrative: the number n =15 frames of virtual frames in the sliding window, and the average playing time T =0.04 second per frame of video frame, i.e. the frame rate of the video is 25 frames per second, when the smoothing time T =15 × 0.04=0.6 second in the sliding window. When the number of the video frames piled up in the buffer queue is 5 frames, 5 frames of video are put into a sliding window, and the total number of the frames in the sliding window is N = N +5=20 frames. Dividing the smoothing time T by the total number of frames N, the average playing time s = T/N =0.03, that is, 5 frames of video frames are piled up, and the playing time of each frame is 0.03 second. In the playing process of the originally accumulated video frames, new video frames can be continuously received, so that accumulated videos exist in the cache queue in a short time, but the playing time of the accumulated videos is shorter than the normal playing time, so that the accumulated video frames can be slowly reduced, the playing time of each frame can gradually approach the normal playing time, namely, the videos are played according to the conventional frame rate, and the smooth playing of live videos after the network card is stopped is realized.
According to the embodiment of the present invention, the obtaining of the average time for playing the video frame specifically includes:
and acquiring the average time of sending the video frames by the live broadcast end.
It should be noted that, in the existing live broadcasting technology, the live broadcasting end may set a recording frame rate of a video frame, and a reciprocal of the recording frame rate is an average time for playing each video frame, that is, an average time for sending the video frame by the live broadcasting end.
According to the embodiment of the present invention, the obtaining of the average time for playing the video frame specifically includes:
and calculating the average time of video frame playing according to the received playing time of the picture group and the total amount of the video frames in the picture group.
It should be noted that, since the receiving end can also adjust the frame rate of watching the video according to the network speed, and when the frame rate is set by the receiving end, the average time for the receiving end to play the video frame can be obtained, another method for obtaining the average time mentioned in the present invention is to calculate the average time for playing the video frame according to the playing duration of the received frame group and the total amount of the video frames in the frame group. It should be added that the processing thread fetches data from the buffer queue and plays the data, and if the timestamp of receiving the video frame is recorded, the average time can also be obtained by taking the quotient of the number of the video frames in a group of frames and the difference between the timestamps of the last frame and the first frame.
According to the embodiment of the present invention, when video frames are accumulated in the buffer queue, the step of placing the accumulated video frames into the smooth window specifically includes:
when the number of the video frames in the buffer queue is 0, receiving video data;
when the number of the video frames in the buffer queue is 1, directly playing the video frames;
and when the number of the video frames in the buffer queue is more than 1, putting the piled video frames into a smooth window.
It should be noted that, when the number of video frames in the buffer queue is 0, it may be the case that the network is stuck, and therefore the video data is not received, and therefore the state of receiving the video data is continuously maintained; when the number of the video frames in the buffer queue is 1, directly playing the video frames, and if the video frames are not received subsequently, the live broadcast picture is the video frame picture until the next video frame is received; when the number of the video frames in the buffer queue is greater than 1, it is indicated that the video frames are piled up, and the playing speed of the video frames needs to be increased, so that the piled up video frames are placed in a smooth window, and then the subsequent steps are performed.
According to the embodiment of the present invention, the step S110 specifically includes: dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time length of the video frame positioned in the first order in the piled video frames;
the step S112 specifically includes: and playing the video frame positioned in the first sequence according to the playing time length, and returning to the step S108.
It should be noted that, only the calculated average playing time is used as the playing time of the first frame in the piled-up video frames, and the first frame video frame is played, and then the process returns to step S108 to continuously determine whether the video frame is still received in the process. It can be understood that if no video frame is received any more, then a frame of video frame has been played, and the total number of frames in the sliding window N ' = N-1, in the view of the data in the above example, the total number of frames in the sliding window is updated to 19 frames, then the playing time length s ' = T/N ' =0.032 of the video frame originally located at the second frame position. In the embodiment of the invention, the best time for playing is calculated for each frame of picture, so that the picture playing effect is smoother.
According to the embodiment of the present invention, when the average time of playing the video frames changes, the process returns to step S102, and the number of the virtual frames is modified.
It can be understood that when the network jitter occurs, the live broadcast software will generally dynamically adjust the video recording and playing frame rate according to the network conditions of the live broadcast end and the playing end, and at this time, when the average time of video frame playing changes, the step S102 is returned to, and the number of virtual frames is adjusted according to the actual frame rate. As to how to set the number of virtual frames under different frame rates, those skilled in the art can set the number according to experience, and the present invention is not limited specifically.
According to the embodiment of the invention, the method further comprises the following steps:
acquiring fluency information fed back by a user side and current network environment information;
inputting the fluency information and the current network environment information into a preset smooth neural network model to obtain first play strategy information;
comparing the first playing strategy information with the current playing strategy to obtain a difference value;
and if the difference degree is greater than a preset difference threshold value, sending a change instruction.
It should be noted that, when the video is played smoothly, the playing strategy may be dynamically adjusted according to the fluency result fed back by the client. The method comprises the steps of firstly, obtaining fluency information fed back by a user side and current network environment information, wherein the fluency information comprises information such as whether a video is jammed or not, whether a green screen appears or not and the like. And then inputting the fluency information and the current network environment information into a preset smooth neural network model, wherein the preset neural network model is trained to obtain first playing strategy information. The first playing strategy information comprises information such as the size of the smooth window parameter, the playing time length and the like. And comparing the first playing strategy information with the current playing strategy to obtain a difference value, wherein the difference value is a comprehensive difference value, namely comparing all the parameters in the difference value, and multiplying the difference value by a preset weight. If the difference degree is larger than the preset difference threshold value, the playing strategy at the moment is possibly greatly different from the strategy obtained by calculation, and a change instruction is sent to change the playing strategy.
The training of the smooth neural network model specifically comprises the following steps:
acquiring fluency information and network environment information of historical time;
preprocessing the fluency information of the historical time and the network environment information to obtain a training sample set;
inputting the training sample set into an initialized neural network model for training;
acquiring the accuracy of an output result;
and if the accuracy is greater than a preset accuracy threshold, stopping training to obtain the smooth neural network model.
It should be noted that, a large amount of historical data and detection results are required in the neural network training process, firstly, fluency information and network environment information of historical time are obtained, and it is worth mentioning that the more the historical data is, the more accurate the model output result is. Preprocessing the fluency information of the historical time and the network environment information to obtain a training sample set, and then inputting the training sample set into an initialized neural network model for training. And judging the accuracy of the trained neural network model, and stopping training if the accuracy exceeds a preset accuracy threshold value to obtain the trained smooth neural network model.
The following is a programming application example for implementing the method of the invention:
(1) initializing the whole device, setting the size of a smoothing window m _ video ConsumeCount, and initializing parameters in the smoothing window.
The videoConsumeCount refers to a smoothing window, the default size of which may be 15, and when the time for receiving a video data frame is too fast or too slow, uneven time needs to be averaged for the next 15 frames to be consumed.
(2) And storing the received video data into a queue, and calculating the average time avgTimePerFrame of each frame through the first gop.
The received video data is put into a cache queue, a processing thread takes the data from the queue and plays the data, a complete video gop is firstly received, the average video frame interval is calculated according to a time stamp, and the first gop data is directly played:
float GopTime = curvideoFrameTimeStamp - firstFrameTimeStamp) * 1000 / 90000;
float avgTimePerFrame = GopTime / videoFrameCount);
(3) and acquiring the number of data in the queue, and calculating the average time consumeTime consumed by each frame in the sliding window for the accumulated video data.
The receiving thread acquires the length currBufferSize of the cache queue, when the currBufferSize =1, the data is directly read out for playing without processing, when the currBufferSize is larger than 1, the data is proved to be accumulated, the time average of normally playing accumulated currBufferSize-1 data is calculated, and the subsequent time consumTime = (currBufferSize-1) (avgTimeFrame/10000: /)/videoConsumount is required to be equally played per frame;
(4) and calculating the time difference between the current frame and the previous frame, and calculating the time point sleepTime of playing the current frame according to the average value, the time difference and the time required to be consumed.
Calculating the time difference value distanceTime = (currtime. QuadPart-lasttime. QuadPart). 1000.00/m _ freq. QuadPart) between the time of taking out the current frame from the thread and the previous frame;
(5) and updating the sliding window and repeating the calculation.
And (3) repeating the steps (2), (3) and (4), and continuously calculating the playing time of each frame according to the number of the data frame buffers to achieve the effect of smoothly playing the video frames.
Fig. 2 shows a block diagram of a video smooth playing system in webcast according to the present invention.
As shown in fig. 2, the present invention discloses a video smooth playing system 2 in live webcast, which includes a memory 21 and a processor 22, where the memory 21 includes a program of a video smooth playing method in live webcast, and when the program of the video smooth playing method in live webcast is executed by the processor 22, the following steps are implemented:
s102, creating a smooth window containing a preset number of virtual frames in a buffer queue;
s104, acquiring the average playing time of the video frames;
s106, calculating the smooth time for playing the video frame in the smooth window according to the product of the average time and the number of the virtual frames;
s108, when video frames are accumulated in the buffer queue, the accumulated video frames are placed in a smooth window;
s110, dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time length of the accumulated video frames;
and S112, playing the video frame according to the playing time length, and returning to the step S108.
It should be noted that, after the live broadcast end receives the video data, the buffering of the video frames is performed in the buffer queue, and the smooth window is set in the buffer queue. The average time of playing video frames, i.e. the reciprocal of the video frame rate, is the dwell time of each video frame picture. The smoothing time refers to the total time of playing the video frames (including the virtual frames) in the smoothing window, that is, the video frames are played in the smoothing time no matter how many frames are in the smoothing window, so the smoothing time is the product of the average time and the number of the virtual frames. When accumulation occurs in the buffer queue, which indicates that a network is unstable before, the video frame which should be received originally and the video frame at the current moment are received into the buffer queue at the same time. And putting the piled video frames into a smooth window, wherein the frames in the smooth window are the sum of the video frames and the virtual frames. And dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time of each frame of the accumulated video frames, namely, the accumulated video frames are subjected to smoothing transition by using the time of the virtual frames. Compared with the method for directly skipping the accumulated video frames or improving the delay playing time in order to ensure the live broadcasting effect in the prior art, the method provided by the invention can achieve the effects of reasonably and effectively processing the uneven video stream and ensuring the smooth playing of the video under the condition of not discarding the video frames and not improving the delay, and solves the problem of poor user viewing experience caused by network blockage.
The following are illustrative: the number n =15 frames of virtual frames in the sliding window, and the average playing time T =0.04 second per frame of video frame, i.e. the frame rate of the video is 25 frames per second, when the smoothing time T =15 × 0.04=0.6 second in the sliding window. When the number of the video frames piled up in the buffer queue is 5 frames, 5 frames of video are put into a sliding window, and the total number of the frames in the sliding window is N = N +5=20 frames. Dividing the smoothing time T by the total number of frames N, the average playing time s = T/N =0.03, that is, 5 frames of video frames are piled up, and the playing time of each frame is 0.03 second. In the playing process of the originally accumulated video frames, new video frames can be continuously received, so that accumulated videos exist in the cache queue in a short time, but the playing time of the accumulated videos is shorter than the normal playing time, so that the accumulated video frames can be slowly reduced, the playing time of each frame can gradually approach the normal playing time, namely, the videos are played according to the conventional frame rate, and the smooth playing of live videos after the network card is stopped is realized.
According to the embodiment of the present invention, the obtaining of the average time for playing the video frame specifically includes:
and acquiring the average time of sending the video frames by the live broadcast end.
It should be noted that, in the existing live broadcasting technology, the live broadcasting end may set a recording frame rate of a video frame, and a reciprocal of the recording frame rate is an average time for playing each video frame, that is, an average time for sending the video frame by the live broadcasting end.
According to the embodiment of the present invention, the obtaining of the average time for playing the video frame specifically includes:
and calculating the average time of video frame playing according to the received playing time of the picture group and the total amount of the video frames in the picture group.
It should be noted that, since the receiving end can also adjust the frame rate of watching the video according to the network speed, and when the frame rate is set by the receiving end, the average time for the receiving end to play the video frame can be obtained, another method for obtaining the average time mentioned in the present invention is to calculate the average time for playing the video frame according to the playing duration of the received frame group and the total amount of the video frames in the frame group. It should be added that the processing thread fetches data from the buffer queue and plays the data, and if the timestamp of receiving the video frame is recorded, the average time can also be obtained by taking the quotient of the number of the video frames in a group of frames and the difference between the timestamps of the last frame and the first frame.
According to the embodiment of the present invention, when video frames are accumulated in the buffer queue, the step of placing the accumulated video frames into the smooth window specifically includes:
when the number of the video frames in the buffer queue is 0, receiving video data;
when the number of the video frames in the buffer queue is 1, directly playing the video frames;
and when the number of the video frames in the buffer queue is more than 1, putting the piled video frames into a smooth window.
It should be noted that, when the number of video frames in the buffer queue is 0, it may be the case that the network is stuck, and therefore the video data is not received, and therefore the state of receiving the video data is continuously maintained; when the number of the video frames in the buffer queue is 1, directly playing the video frames, and if the video frames are not received subsequently, the live broadcast picture is the video frame picture until the next video frame is received; when the number of the video frames in the buffer queue is greater than 1, it is indicated that the video frames are piled up, and the playing speed of the video frames needs to be increased, so that the piled up video frames are placed in a smooth window, and then the subsequent steps are performed.
According to the embodiment of the present invention, the step S110 specifically includes: dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time length of the video frame positioned in the first order in the piled video frames;
the step S112 specifically includes: and playing the video frame positioned in the first sequence according to the playing time length, and returning to the step S108.
It should be noted that, only the calculated average playing time is used as the playing time of the first frame in the piled-up video frames, and the first frame video frame is played, and then the process returns to step S108 to continuously determine whether the video frame is still received in the process. It can be understood that if no video frame is received any more, then a frame of video frame has been played, and the total number of frames in the sliding window N ' = N-1, in the view of the data in the above example, the total number of frames in the sliding window is updated to 19 frames, then the playing time length s ' = T/N ' =0.032 of the video frame originally located at the second frame position. In the embodiment of the invention, the best time for playing is calculated for each frame of picture, so that the picture playing effect is smoother.
According to the embodiment of the present invention, when the average time of playing the video frames changes, the process returns to step S102, and the number of the virtual frames is modified.
It can be understood that when the network jitter occurs, the live broadcast software will generally dynamically adjust the video recording and playing frame rate according to the network conditions of the live broadcast end and the playing end, and at this time, when the average time of video frame playing changes, the step S102 is returned to, and the number of virtual frames is adjusted according to the actual frame rate. As to how to set the number of virtual frames under different frame rates, those skilled in the art can set the number according to experience, and the present invention is not limited specifically.
According to the embodiment of the invention, the method further comprises the following steps:
acquiring fluency information fed back by a user side and current network environment information;
inputting the fluency information and the current network environment information into a preset smooth neural network model to obtain first play strategy information;
comparing the first playing strategy information with the current playing strategy to obtain a difference value;
and if the difference degree is greater than a preset difference threshold value, sending a change instruction.
It should be noted that, when the video is played smoothly, the playing strategy may be dynamically adjusted according to the fluency result fed back by the client. The method comprises the steps of firstly, obtaining fluency information fed back by a user side and current network environment information, wherein the fluency information comprises information such as whether a video is jammed or not, whether a green screen appears or not and the like. And then inputting the fluency information and the current network environment information into a preset smooth neural network model, wherein the preset neural network model is trained to obtain first playing strategy information. The first playing strategy information comprises information such as the size of the smooth window parameter, the playing time length and the like. And comparing the first playing strategy information with the current playing strategy to obtain a difference value, wherein the difference value is a comprehensive difference value, namely comparing all the parameters in the difference value, and multiplying the difference value by a preset weight. If the difference degree is larger than the preset difference threshold value, the playing strategy at the moment is possibly greatly different from the strategy obtained by calculation, and a change instruction is sent to change the playing strategy.
The training of the smooth neural network model specifically comprises the following steps:
acquiring fluency information and network environment information of historical time;
preprocessing the fluency information of the historical time and the network environment information to obtain a training sample set;
inputting the training sample set into an initialized neural network model for training;
acquiring the accuracy of an output result;
and if the accuracy is greater than a preset accuracy threshold, stopping training to obtain the smooth neural network model.
It should be noted that, a large amount of historical data and detection results are required in the neural network training process, firstly, fluency information and network environment information of historical time are obtained, and it is worth mentioning that the more the historical data is, the more accurate the model output result is. Preprocessing the fluency information of the historical time and the network environment information to obtain a training sample set, and then inputting the training sample set into an initialized neural network model for training. And judging the accuracy of the trained neural network model, and stopping training if the accuracy exceeds a preset accuracy threshold value to obtain the trained smooth neural network model.
A third aspect of the present invention provides a computer-readable storage medium, where the computer-readable storage medium includes a program of a video smooth playing method in live webcasting, and when the program of the video smooth playing method in live webcasting is executed by a processor, the method implements the steps of the video smooth playing method in live webcasting as described in any one of the above.
The invention discloses a video smooth playing method, a system and a readable storage medium in network live broadcast.A smooth window containing a preset number of virtual frames is established in a buffer queue, the smooth time for playing the video frames in the smooth window is calculated according to the product of the playing of the video frames and the number of the virtual frames, when the video frames are accumulated in the buffer queue, the accumulated video frames are put into the smooth window, the smooth time is divided by the number of the frames in the smooth window to obtain the playing time of the accumulated video frames, and the video frames are played according to the playing time, so that the non-uniform video stream is reasonably and effectively processed under the condition of not discarding the video frames and not improving the time delay, the effect of smooth playing of the video is ensured, and the problem of poor viewing user experience caused by a network card is solved.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, all the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
Alternatively, the integrated unit of the present invention may be stored in a computer-readable storage medium if it is implemented in the form of a software functional module and sold or used as a separate product. Based on such understanding, the technical solutions of the embodiments of the present invention may be essentially implemented or a part contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a RAM, a magnetic or optical disk, or various other media that can store program code.
Claims (6)
1. A video smooth playing method in network live broadcast is characterized by comprising the following steps:
s102, creating a smooth window containing a preset number of virtual frames in a buffer queue;
s104, acquiring the average playing time of the video frames;
s106, calculating the smooth time for playing the video frame in the smooth window according to the product of the average time and the number of the virtual frames;
s108, when video frames are accumulated in the buffer queue, the accumulated video frames are placed in a smooth window;
s110, dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time length of the accumulated video frames;
s112, playing the video frame according to the playing duration, and returning to the step S108;
the obtaining of the average time for playing the video frame specifically includes:
calculating the average time of video frame playing according to the received playing time of the picture group and the total amount of the video frames in the picture group;
when the video frames are accumulated in the buffer queue, the step of placing the accumulated video frames into the smoothing window specifically includes:
when the number of the video frames in the buffer queue is 0, receiving video data;
when the number of the video frames in the buffer queue is 1, directly playing the video frames;
when the number of the video frames in the buffer queue is more than 1, putting the piled video frames into a smooth window;
acquiring fluency information fed back by a user side and current network environment information;
inputting the fluency information and the current network environment information into a preset smooth neural network model to obtain first play strategy information;
comparing the first playing strategy information with the current playing strategy to obtain a difference value;
and if the difference degree is greater than a preset difference threshold value, sending a change instruction.
2. The method for smoothly playing the video in the live webcast as claimed in claim 1, wherein the obtaining the average time of playing the video frames specifically comprises:
and acquiring the average time of sending the video frames by the live broadcast end.
3. The method as claimed in claim 1, wherein the video playing method includes,
step S110 specifically includes: dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time length of the video frame positioned in the first order in the piled video frames;
step S112 specifically includes: and playing the video frame positioned in the first sequence according to the playing time length, and returning to the step S108.
4. A video smooth playing system in network living broadcast is characterized by comprising a memory and a processor, wherein the memory comprises a video smooth playing method program in network living broadcast, and the video smooth playing method program in network living broadcast realizes the following steps when being executed by the processor:
s102, creating a smooth window containing a preset number of virtual frames in a buffer queue;
s104, acquiring the average playing time of the video frames;
s106, calculating the smooth time for playing the video frame in the smooth window according to the product of the average time and the number of the virtual frames;
s108, when video frames are accumulated in the buffer queue, the accumulated video frames are placed in a smooth window;
s110, dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time length of the accumulated video frames;
s112, playing the video frame according to the playing duration, and returning to the step S108;
the obtaining of the average time for playing the video frame specifically includes:
calculating the average time of video frame playing according to the received playing time of the picture group and the total amount of the video frames in the picture group;
when the video frames are accumulated in the buffer queue, the step of placing the accumulated video frames into the smoothing window specifically includes:
when the number of the video frames in the buffer queue is 0, receiving video data;
when the number of the video frames in the buffer queue is 1, directly playing the video frames;
when the number of the video frames in the buffer queue is more than 1, putting the piled video frames into a smooth window;
acquiring fluency information fed back by a user side and current network environment information;
inputting the fluency information and the current network environment information into a preset smooth neural network model to obtain first play strategy information;
comparing the first playing strategy information with the current playing strategy to obtain a difference value;
and if the difference degree is greater than a preset difference threshold value, sending a change instruction.
5. The system of claim 4, wherein the video smooth playing system in network live broadcast,
step S110 specifically includes: dividing the smoothing time by the number of frames in the smoothing window to obtain the playing time length of the video frame positioned in the first order in the piled video frames;
step S112 specifically includes: and playing the video frame positioned in the first sequence according to the playing time length, and returning to the step S108.
6. A computer-readable storage medium, characterized in that the computer-readable storage medium includes a program of a video smooth playing method in live webcasting, and when the program of the video smooth playing method in live webcasting is executed by a processor, the steps of a video smooth playing method in live webcasting according to any one of claims 1 to 3 are implemented.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110799460.5A CN113271496B (en) | 2021-07-15 | 2021-07-15 | Video smooth playing method and system in network live broadcast and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110799460.5A CN113271496B (en) | 2021-07-15 | 2021-07-15 | Video smooth playing method and system in network live broadcast and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113271496A CN113271496A (en) | 2021-08-17 |
| CN113271496B true CN113271496B (en) | 2021-10-01 |
Family
ID=77236567
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110799460.5A Active CN113271496B (en) | 2021-07-15 | 2021-07-15 | Video smooth playing method and system in network live broadcast and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113271496B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104394450A (en) * | 2014-11-14 | 2015-03-04 | 青岛海信宽带多媒体技术有限公司 | Multimedia-frame processing method and device |
| CN105898536A (en) * | 2016-04-29 | 2016-08-24 | 乐视控股（北京）有限公司 | Video play method and device |
| CN107371061A (en) * | 2017-08-25 | 2017-11-21 | 普联技术有限公司 | A kind of video stream playing method, device and equipment |
| CN108965807A (en) * | 2018-07-17 | 2018-12-07 | 深圳市共进电子股份有限公司 | Control method for playing back, device, terminal and the storage medium of real-time video |
| CN111294634A (en) * | 2020-02-27 | 2020-06-16 | 腾讯科技（深圳）有限公司 | Live broadcast method, device, system, equipment and computer readable storage medium |
| CN111918142A (en) * | 2020-07-29 | 2020-11-10 | 杭州叙简科技股份有限公司 | Smoothing method, device, equipment and medium for converting national standard video code stream into RTP stream |
| CN112702649A (en) * | 2019-10-22 | 2021-04-23 | 高新兴科技集团股份有限公司 | Live video playing fluency optimization method of mobile terminal, storage medium and electronic equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104702968B (en) * | 2015-02-17 | 2019-06-11 | 华为技术有限公司 | A kind of video frame frame losing method and video transmission device |
-
2021
- 2021-07-15 CN CN202110799460.5A patent/CN113271496B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104394450A (en) * | 2014-11-14 | 2015-03-04 | 青岛海信宽带多媒体技术有限公司 | Multimedia-frame processing method and device |
| CN105898536A (en) * | 2016-04-29 | 2016-08-24 | 乐视控股（北京）有限公司 | Video play method and device |
| CN107371061A (en) * | 2017-08-25 | 2017-11-21 | 普联技术有限公司 | A kind of video stream playing method, device and equipment |
| CN108965807A (en) * | 2018-07-17 | 2018-12-07 | 深圳市共进电子股份有限公司 | Control method for playing back, device, terminal and the storage medium of real-time video |
| CN112702649A (en) * | 2019-10-22 | 2021-04-23 | 高新兴科技集团股份有限公司 | Live video playing fluency optimization method of mobile terminal, storage medium and electronic equipment |
| CN111294634A (en) * | 2020-02-27 | 2020-06-16 | 腾讯科技（深圳）有限公司 | Live broadcast method, device, system, equipment and computer readable storage medium |
| CN111918142A (en) * | 2020-07-29 | 2020-11-10 | 杭州叙简科技股份有限公司 | Smoothing method, device, equipment and medium for converting national standard video code stream into RTP stream |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113271496A (en) | 2021-08-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10110650B2 (en) | Client side stream switching | |
| US9167007B2 (en) | Stream complexity mapping | |
| US9503491B2 (en) | Playback stall avoidance in adaptive media streaming | |
| EP2300928B1 (en) | Client side stream switching | |
| US20220232222A1 (en) | Video data processing method and apparatus, and storage medium | |
| US20070217505A1 (en) | Adaptive Decoding Of Video Data | |
| CN104967884B (en) | A kind of bitstreams switching method and apparatus | |
| EP3448041A1 (en) | Video player client, system, and method for live broadcast video synchronization | |
| WO2011146898A2 (en) | Internet system for ultra high video quality | |
| US11863841B2 (en) | Video playing control method and system | |
| CN109168083A (en) | A kind of Streaming Media real time playing method and device | |
| AU2018275194A1 (en) | Temporal placement of a rebuffering event | |
| CN110139128B (en) | Information processing method, interceptor, electronic equipment and storage medium | |
| CN113766261A (en) | Method and device for determining pre-pulling duration, electronic equipment and storage medium | |
| CN113271496B (en) | Video smooth playing method and system in network live broadcast and readable storage medium | |
| TWI723394B (en) | Method for shaping video streams, set-up box and storage medium | |
| EP4104451B1 (en) | Low latency content delivery | |
| CN115348409A (en) | Video data processing method and device, terminal equipment and storage medium | |
| EP3664456A1 (en) | Apparatus and method for playing streamed media | |
| CN117440209B (en) | Implementation method and system based on singing scene | |
| JP7350866B2 (en) | Display control device, transmission device, display control method and program | |
| CN114095763B (en) | Video list pushing method and play quality assessment method | |
| EP2739061A1 (en) | Multi resolutions adaptive video summarization and its adaptive delivery | |
| US20220400309A1 (en) | Engagement estimation apparatus, engagement estimation method and program | |
| CN117729376A (en) | Video playing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |