CN115942048A

CN115942048A - Video playing method and device, electronic equipment and storage medium

Info

Publication number: CN115942048A
Application number: CN202211675397.5A
Authority: CN
Inventors: 赵恩军; 肖集华; 翟小刚; 刁磊
Original assignee: Suzhou Keda Special Video Co ltd; Suzhou Keda Technology Co Ltd
Current assignee: Suzhou Keda Special Video Co ltd; Suzhou Keda Technology Co Ltd
Priority date: 2022-12-26
Filing date: 2022-12-26
Publication date: 2023-04-07

Abstract

The invention relates to the technical field of video processing, in particular to a video playing method, a video playing device, electronic equipment and a storage medium, wherein the method comprises the steps of obtaining a current frame rate to determine a current frame processing reference value; acquiring the number of video frames in a frame buffer area, and determining a current frame processing correction value based on the size relation between the number of the video frames and a preset number; determining a current frame processing interval based on the sum of the current frame processing reference value and the current frame processing correction value; and extracting the current frame from the frame buffer area based on the current frame processing interval to perform target processing to play the processed video frame, wherein the target processing comprises decoding processing and/or playing processing. Frame buffering before target processing is carried out by utilizing a frame buffer area, and meanwhile, a frame processing correction value is determined by combining the size relation between the number of video frames in the frame buffer area and the preset number when a current frame is extracted, so that the target processing time interval between two adjacent frames is ensured to be within a controllable range, and the aim of smooth playing is fulfilled.

Description

Video playing method and device, electronic equipment and storage medium

Technical Field

The invention relates to the technical field of video processing, in particular to a video playing method and device, electronic equipment and a storage medium.

Background

In recent years, the information technology is developed at a high speed, terminals suitable for teleconferencing are widely applied, the attention degree of the countries to the development of the information technology in recent years is reflected from the traditional teleconferencing which is in the same physical meeting room to the current teleconferencing which is not limited by physical space, and the development brings more convenience to the work and life of people.

However, in the encoding, decoding and transmission processes, the jitter of the code stream is inevitably caused, and the fast and slow pause is caused. First, when the far-end encoding and the local-end decoding of the video conference are carried out, due to the performance of the device, the time of each frame is unequal, for example, 60 frames of video, the time interval of each frame should be 16.67ms, but as shown in fig. 1, due to the instability of the codec, the encoded interval is between 10 and 90ms, and the decoded interval is between 1 and 100ms, or even more as shown in fig. 3. Although the total frame rate is 60 frames counted per second, the difference between the frame intervals is too large, so that the effect of being very obvious jammed and suddenly slow can be achieved during playing. Secondly, in the network transmission process, because the network state is not controllable, the time interval of each received frame is usually unstable, especially when the I frame is in a poor state, jitter can reach 10-70ms, and even more as shown in fig. 2.

Since jitter is unavoidable, it becomes critical how to digest the unsmooth playback from such jitter.

Disclosure of Invention

In view of this, embodiments of the present invention provide a video playing method, an apparatus, an electronic device, and a storage medium, so as to solve the problem of unsmooth video playing caused by jitter.

According to a first aspect, an embodiment of the present invention provides a video playing method, including:

acquiring a current frame rate to determine a current frame processing reference value;

acquiring the number of video frames in a frame buffer area, and determining a current frame processing correction value based on the size relation between the number of the video frames and a preset number;

determining a current frame processing interval based on the sum of the current frame processing reference value and the current frame processing correction value;

and extracting the current frame from the frame buffer area based on the current frame processing interval to perform target processing so as to play the processed video frame, wherein the target processing comprises decoding processing and/or playing processing.

The video playing method provided by the embodiment of the invention utilizes the frame buffer area to buffer the frames before target processing, and simultaneously determines the frame processing correction value by combining the size relation between the number of the video frames in the frame buffer area and the preset number when the current frame is extracted, so as to ensure that the obtained current frame processing interval can accord with the actual buffer condition in the frame buffer area, and extracts the current frame from the frame buffer area based on the current frame processing interval to carry out target processing, so as to ensure that the target processing time interval between two adjacent frames is in a controllable range, thereby achieving the purpose of smooth playing.

In some embodiments, the determining a current frame processing correction value based on a size relationship between the number of video frames and a preset number includes:

calculating the difference value between the video frame and the preset number, and determining the number difference;

when the number difference is less than zero, determining a first correction value which is greater than zero as the current frame processing correction value;

determining a second correction value that is less than zero as the current frame processing correction value when the number difference is greater than zero.

In the video playing method provided by the embodiment of the invention, under the condition that the number difference is less than zero, the number of video frames in the frame buffer area is less, and slow processing is required, so that a first correction value which is greater than zero is used as a current frame processing correction value; when the number difference is greater than zero, it indicates that the number of video frames in the frame buffer is large and fast processing is required, and therefore, a second correction value smaller than zero is used as the current frame processing correction value. The slow processing or the fast processing is realized through the two conditions so as to ensure that the target processing time interval between any two adjacent frames is within a controllable range.

In some embodiments, said determining a first correction value greater than zero as said current frame processing correction value when said number difference is less than zero comprises:

determining a first sub-correction value greater than zero as the current frame processing correction value when the number difference is greater than a first value and less than zero, and determining a second sub-correction value greater than the first sub-correction value as the current frame processing correction value when the number difference is less than or equal to the first value;

and/or the presence of a gas in the gas,

when the number difference is greater than zero, determining a first correction value greater than zero as the current frame processing correction value includes:

determining a third sub-correction value that is less than zero as the current frame processing correction value when the number difference is greater than zero and less than a second value, and determining a fourth sub-correction value that is less than the third sub-correction value as the current frame processing correction value when the number difference is greater than or equal to the second value.

The video playing method provided by the embodiment of the invention equally divides the number difference into different time thresholds to process the correction value under the condition that the number difference is larger than zero and the number difference is smaller than zero, thereby realizing refined time control and further ensuring the fluency of video playing.

In some embodiments, the extracting a current frame from the frame buffer based on the current frame processing interval for target processing to play a processed video frame includes:

acquiring the starting time of the target processing and the processing relative time of the previous frame, wherein the processing relative time is obtained by accumulating the frame processing interval from the first frame of the target processing to the previous frame;

accumulating the processing interval of the current frame to the processing relative time of the previous frame to obtain the processing relative time of the current frame;

accumulating the starting time and the processing relative time of the current frame to obtain the target processing time of the current frame;

and when the current time reaches the target processing time of the current frame, extracting the current frame from the frame buffer area to perform the target processing so as to play the processed video frame.

According to the video playing method provided by the embodiment of the invention, as the starting time is a fixed time value, the target processing time is judged by the accumulation of the starting time of the target processing and the processing relative time, and the accuracy of the target processing time is ensured.

In some embodiments, in the case that the target process is a decoding process, the obtaining a start time of the target process includes:

judging whether the frame numbers of the video frames are continuous or not;

and when the frame numbers are not continuous, determining the time of the video frame with the discontinuous frame number appearing for the first time as the starting time of the target processing.

The video playing method provided by the embodiment of the invention determines whether the decoding processing is performed for the new time according to the continuity of the frame numbers of the video frames, thereby simplifying the reliability of the determination of the starting time.

In some embodiments, the method further comprises:

judging whether frame control time is abnormal, wherein the frame control time comprises at least one of the current time, the starting time and the processing relative time;

and when the frame control time is abnormal, resetting the starting time and the processing relative time.

The video playing method provided by the embodiment of the invention resets the starting time and the processing relative time under the condition that the frame control time is abnormal, but the target processing time is not influenced by the resetting processing, and the reliability of the target processing is improved under the condition that the target processing is not influenced.

In some embodiments, in a case that the target process is a decoding process, the obtaining the current frame rate includes:

judging whether the time for starting decoding exceeds the preset time or not;

and when the time for starting decoding does not exceed the preset time, determining a preset frame rate as the current frame rate.

The video playing method provided by the embodiment of the invention indicates that the frame rate can not be counted at the moment when the preset time is not reached, and the preset frame rate is determined as the current frame rate under the condition so as to ensure the normal operation of the target processing.

According to a second aspect, an embodiment of the present invention further provides a video playing apparatus, including:

the first obtaining module is used for obtaining the current frame rate so as to determine a current frame processing reference value;

the second acquisition module is used for acquiring the number of video frames in a frame buffer area and determining a current frame processing correction value based on the size relation between the number of the video frames and the preset number;

a determining module, configured to determine a current frame processing interval based on a sum of the current frame processing reference value and the current frame processing correction value;

and the processing module is used for extracting the current frame from the frame buffer area based on the current frame processing interval to perform target processing so as to play the processed video frame, wherein the target processing comprises decoding processing and/or playing processing.

According to a third aspect, an embodiment of the present invention provides an electronic device, including: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing therein computer instructions, and the processor executing the computer instructions to perform the video playing method according to the first aspect or any one of the embodiments of the first aspect.

According to a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions for causing a computer to execute the video playing method described in the first aspect or any one implementation manner of the first aspect.

It should be noted that, for corresponding beneficial effects of the video playing apparatus, the electronic device and the computer readable storage medium provided in the embodiment of the present invention, please refer to the description of the corresponding beneficial effects of the video playing method above, which is not described herein again.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.

FIG. 1 is a schematic diagram of coding per frame time interval statistics;

FIG. 2 is a schematic illustration of the received per-frame time interval statistics before decoding;

FIG. 3 is a schematic diagram of decoding each frame time interval;

FIG. 4 is a schematic illustration of the time interval statistics of the post-decoding smoothing process;

fig. 5 is a flowchart of a video playing method according to an embodiment of the present invention;

fig. 6 is a flowchart of a video playing method according to an embodiment of the present invention;

fig. 7 is a flowchart of a video playing method according to an embodiment of the present invention;

fig. 8 is a flowchart of a video playing method according to an embodiment of the present invention;

FIG. 9 is a diagram illustrating time interval statistics after video playback processing according to an embodiment of the present invention;

fig. 10 is a block diagram of a video playback apparatus according to an embodiment of the present invention;

fig. 11 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In the related technology, a passive buffering mechanism is adopted for processing video playing, a layer of buffering is added at a code stream receiving position, and when excessive receiving is carried out at a certain time, frames are put into a buffer area to avoid frame loss caused by decoding failure; when reception is too slow, no extra processing is done. This method is more resistant to packet loss and not very resistant to jitter. In addition, the above method has two disadvantages, one is that it can avoid the fast broadcasting, although the packet will not be lost when the frame comes too fast. When the frame comes too slowly, the situation that no frame can be played occurs, resulting in pause. Secondly, only adding buffer at the receiving part can only act on the jitter brought by coding and network, and has no effect on the jitter brought by decoding.

Based on this, the video playing method provided by the embodiment of the present invention is applicable to: the frame rate of encoding, decoding and playing can reach the preset frame rate, but the video pause phenomenon still exists. It should be noted that the disclosure does not address the stuck caused by insufficient codec or output frame rate.

Specifically, the video conference system at least comprises a far end and a local end, wherein the far end encodes a video collected by the camera, and then sends the video to the local end through a network, the local end decodes the video, and then the decoded video is sent to the playing module to be played. The video playing method provided by the embodiment of the invention mainly processes at the receiving end of the processing chain, so that the video can be played more smoothly.

If the video is to be played smoothly, assuming that the frame rate of the video is 60 frames, the presentation time of each frame should be 1000ms/60=16.67ms, and if the presentation time interval of each frame can be precisely controlled to be 16.67ms, the video is necessarily smooth. However, in a non-real-time operating system, it is not practical to strictly and accurately control the playing time of each frame, so that it is necessary to smooth the playing time as much as possible, and the time difference of each frame is within an acceptable range, so that human eyes cannot see the playing time of each frame. Based on this idea, in the scheme provided by the embodiment of the present invention, a concept of a frame buffer is added, and the frame buffer is added before the current frame is subjected to target processing. Wherein the target process may be at least one of a decoding process and a playing process. That is, the frame buffer may be added only before decoding, may be added only before playback, or may be added both before decoding and before playback.

It should be noted that the definition of decoding in the embodiment of the present invention includes smoothing processing and decoding processing before decoding; the definition of playback includes smoothing processing before playback and playback processing.

For example, if a frame buffer is added before decoding, it is referred to as a decoding buffer. As described above, when the stream reaches the decoder process, the queue of the stream cannot be maintained, but the stream is fast or slow, and the stream comes in a group, which is not desirable for the playback end. The playout end expects that each frame comes at a fixed time interval, so adding a pre-decode frame buffer forces it to maintain queue again. The method specifically comprises the following steps: after the frame to be decoded is taken, the frame is not directly sent to a decoder, but is firstly put into a decoding buffer area, a time control means is added, so that the difference value of the time interval between every two frames from the decoding frame buffer area is within the preset time, and then the frame is sent to the decoder. The preset time interval is set according to actual requirements, and is not limited herein.

If a frame buffer is added before playing, it is called a play buffer. Because a plurality of tasks/processes in the playing end run simultaneously, a plurality of threads run simultaneously in the same process, and the busy playing end and the decoder cause that the time spent by the decoder for decoding a frame is not even, the time interval for decoding a frame can even be from 1ms to 40ms, and the difference between the preset time and the time interval is very large, so that human eyes can see the existence of the pause, and the experience effect is influenced. Therefore, the decoded frame is not directly sent to the playing module, but is put into a playing buffer area, and a time control means is added to ensure that the difference value of the time interval between each frame from the frame buffer area is within the preset time and then sent to the playing module.

The video playing method provided by the embodiment of the invention adopts multi-level buffering, optimizes fluency indexes to a greater extent under the condition of controllable and acceptable delay, and improves the overall experience to a greater extent.

In accordance with an embodiment of the present invention, there is provided a video playback method embodiment, it should be noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than here.

In this embodiment, a video playing method is provided, which can be used in electronic devices, such as computers, mobile terminals, and the like, fig. 5 is a flowchart of the video playing method according to the embodiment of the present invention, as shown in fig. 5, the flowchart includes the following steps:

and S11, acquiring the current frame rate to determine a current frame processing reference value.

As described above, if the video is to be played smoothly, assuming that the frame rate of the video is 60 frames, the presentation time of each frame should be 1000ms/60=16.67ms. Based on this, 16.67ms is referred to as a frame processing reference value.

Wherein, the frame rate detection is real-time detection. However, it should be noted that, because the electronic device only supports several specific frame rates, there is a certain window for detection in the frame rate detection logic, and therefore, the frame rate does not jump back and forth.

S12, acquiring the number of video frames in a frame buffer area, and determining a current frame processing correction value based on the size relation between the number of the video frames and the preset number.

After receiving the video frame, the video frame is added into a frame buffer area, and the calculation of the current frame processing correction value is carried out before the target processing is carried out each time. The principle of determining the current frame processing correction value is as follows: and determining the size of the current frame processing correction value according to the size relation between the video frames in the frame buffer area and the preset number. If the number of the video frames is larger than the preset number, the situation of frame backlog exists, the processing needs to be accelerated, correspondingly, the frame processing interval needs to be reduced by numerical value reduction on the basis of a frame processing reference value, and based on the frame processing reference value, the current frame processing correction value needs to be smaller than zero; conversely, the current frame processing correction value needs to be greater than zero.

If the number of the video frames in the frame buffer area is consistent with the preset number, the current frame processing reference value is appropriate, and based on the reference value, the current frame processing correction value is zero.

Of course, the above-mentioned setting of the current frame processing correction value is only an example, and particularly, the setting needs to be performed according to actual requirements.

And S13, determining the current frame processing interval based on the sum of the current frame processing reference value and the current frame processing correction value.

The current frame processing reference value is determined based on the current frame rate, and the current frame processing correction value is determined based on the size relation between the number of the video frames in the frame buffer and the preset number. On the basis, the current frame processing interval is the sum of the current frame processing reference value and the current frame processing correction value.

For example, the video frames in the frame buffer are respectively video frame 1 to video frame 5 according to the sequence. Then the current frame is the first video frame in the frame buffer, i.e. video frame 1.

After extracting the video frame 0 preceding the video frame 1 from the frame buffer, the time control for the current frame is started. Namely, the current frame processing reference value and the current frame processing correction value are determined, and then the two values are added to obtain the current frame processing interval.

Continuing with the above example, the current frame processing interval is the time interval between the extraction of video frame 1 and the extraction of video frame 0.

And S14, extracting the current frame from the frame buffer area based on the current frame processing interval to perform target processing so as to play the processed video frame.

Wherein the target process comprises a decoding process and/or a playing process.

Continuing with the above example, it may be that each time a video frame is extracted from the frame buffer, the extraction timestamp is recorded and timed. And comparing the timing time with the current frame processing time interval obtained by calculation, if the timing duration reaches the current frame processing time interval, indicating that the current frame processing time is reached, and extracting the current frame from the frame buffer area for target processing. After the target processing is performed on the current frame, the processed video frame is played.

As described above, the target process may be a decoding process, a playback process, or a decoding process and a playback process. Correspondingly, adding a decoding frame buffer area before decoding, and carrying out time control on the video frame in the decoding frame buffer area, namely, carrying out smoothing processing on the video frame by using the method of the previous step before decoding, then decoding the video frame subjected to the smoothing processing of the previous step, and finally directly playing the decoded video frame; or, before playing, setting a playing frame buffer area, and performing time control on the video frame in the playing frame buffer area, namely, before playing, performing smoothing processing on the decoded video frame by using the method in the previous step, and then playing the video frame subjected to smoothing processing in the previous step; or, setting a decoding frame buffer area and a playing frame buffer area respectively, and performing time control respectively; the method comprises the steps of firstly smoothing the video frame by adopting the method of the steps before decoding, then decoding the processed video frame, and playing the smoothed video frame before playing instead of directly playing the video frame after decoding.

Taking target processing as decoding processing and playing processing as an example, respectively setting a decoding frame buffer area and a playing frame buffer area, correspondingly, the preset number corresponding to the decoding processing is called as a first preset number, and the obtained current frame processing correction value is a current frame processing first correction value; the preset number corresponding to the playing process is called a second preset number, and the obtained current frame processing correction value is a current frame processing second correction value.

Specifically, the frame buffer is set before decoding or playing, which is performed according to actual requirements, and is not limited herein.

In the video playing method provided by this embodiment, a frame buffer area is used for frame buffering before target processing, and a frame processing correction value is determined according to a size relationship between the number of video frames in the frame buffer area and a preset number when a current frame is extracted, so that an obtained current frame processing interval can meet an actual buffer condition in the frame buffer area, and the current frame is extracted from the frame buffer area for target processing based on the current frame processing interval, so that a target processing time interval between two adjacent frames is ensured to be within a controllable range, and the purpose of smooth playing is achieved.

In this embodiment, a video playing method is provided, which can be used in electronic devices, such as computers, mobile terminals, and the like, fig. 6 is a flowchart of the video playing method according to the embodiment of the present invention, and as shown in fig. 6, the flowchart includes the following steps:

and S21, acquiring the current frame rate to determine a current frame processing reference value.

For frame control before decoding, the current frame rate can be obtained by real-time detection; if the frame control before playing is set at the same time, the current frame rate determined before decoding can be applied to the frame control before playing.

In the case where the target process is a decoding process, acquiring the current frame rate includes:

(1) And judging whether the time for starting decoding exceeds the preset time.

(2) And when the time for starting decoding does not exceed the preset time, determining the preset frame rate as the current frame rate.

The preset time is a time interval at which the frame rate can be determined, for example, 1S. And if the time for starting decoding does not reach the preset time, determining the preset frame rate as the current frame rate. And if the time for starting decoding reaches the preset time, counting the number of the received video frames in a period of time, and determining the current frame rate based on the counting result. For example, the number of received video frames in 1S is counted, and the number is taken as the current frame rate.

And when the preset time is not reached, the frame rate cannot be counted at the moment, and in this case, the preset frame rate is determined as the current frame rate to ensure the normal operation of the target processing.

For the rest, please refer to S11 in the embodiment shown in fig. 5, which is not described herein again.

S22, acquiring the number of the video frames in the frame buffer area, and determining the current frame processing correction value based on the size relation between the number of the video frames and the preset number.

Specifically, the above S22 includes:

s221, acquiring the number of the video frames in the frame buffer area.

For the frame buffer area, a counter can be set, and if a frame of video frame is newly added, the counting result is added by 1; and if a frame of video frame is newly extracted, subtracting 1 from the counting result. And determining the number of the video frames in the frame buffer area according to the counting result.

S222, calculating the difference value between the video frame and the preset number, and determining the number difference.

The preset number is set according to actual requirements, for example, for a decoding frame buffer set before decoding, the preset number is set to be 1 or 3 or 4, and the like; the preset number of the play frame buffers set before playing is set to 5 or 6, and so on. Different preset quantities are respectively set according to different target processing so as to meet different processing requirements.

And calculating the difference value between the video frame and the preset number to obtain a number difference, wherein the number difference can represent the current state of the frame buffer area. The current state includes too many frames, normal, and too few frames.

And S223, when the number difference is less than zero, determining the first correction value which is greater than zero as the current frame processing correction value.

And S224, when the quantity difference is larger than zero, determining a second correction value smaller than zero as the current frame processing correction value.

The number difference is less than zero, which indicates that the number of video frames in the frame buffer is less than the preset number, and at this time, slow play needs to be performed. That is, it is necessary to delay the current frame processing correction value and play it back on the basis of the current frame processing reference value. Since the delay is to be implemented, the current frame processing correction value is greater than zero. Therefore, the first correction value larger than zero is determined as the current frame processing correction value.

The number difference is greater than zero, which indicates that the number of video frames in the frame buffer is greater than the preset number, and at this time, fast play needs to be performed. That is, it is necessary to reduce the current frame processing correction value and play the current frame processing correction value based on the current frame processing reference value. Since the fast play is to be realized, the current processing correction value is less than zero. Therefore, the second correction value smaller than zero is determined as the current frame processing correction value.

Further, to achieve finer temporal control over the video frames in the frame buffer, different thresholds may be set for the number difference greater than zero and the number difference less than zero, respectively, to further divide the intervals greater than zero and to further divide the intervals less than zero. The number of divisions of the interval greater than zero and the number of divisions of the interval less than zero are set according to actual requirements, and are not limited herein.

For the condition that the number difference is greater than zero, the current frame processing correction value is smaller than zero in the condition, and the current frame processing correction value corresponding to the interval which is farther away from zero is smaller; the section closer to zero corresponds to a larger current frame processing correction value.

For the case that the number difference is less than zero, the current frame processing correction value is greater than zero in this case, and the current frame processing correction value corresponding to the interval farther away from zero is larger; the closer the interval is to zero, the smaller the current frame processing correction value.

In some embodiments, the interval with the number difference smaller than zero is divided into 2 intervals, and the intervals are distinguished by a first numerical value. Based on this, the above S223 includes:

(1) And when the number difference is larger than the first value and smaller than zero, determining the first sub-correction value larger than zero as the current frame processing correction value.

(2) When the number difference is less than or equal to the first value, a second sub-correction value larger than the first sub-correction value is determined as the current frame processing correction value.

For example, taking the target process as the playback process, i.e., performing the smoothing process before playback, the difference in the amount is represented by Diff, and the first value is-2. Based on this, if Diff < = -2, it indicates that the number in the buffer is lower than the base number, at this time, slow play needs to be performed, and the second sub-correction value is 4ms; if-2 instead of diff < =0, slow play is also performed, and the first sub-correction value is 2ms.

In some embodiments, the interval with the number difference larger than zero is divided into 2 intervals and distinguished by the second numerical value. The S224 includes:

(1) And when the quantity difference is larger than zero and smaller than a second value, determining a third sub-correction value smaller than zero as the current frame processing correction value.

(2) And determining a fourth sub-correction value smaller than the third sub-correction value as the current frame processing correction value when the number difference is greater than or equal to the second value.

If the second value is 2, based on this, if Diff > =2, it indicates that the number of frames in the buffer is excessive, at this time, fast play needs to be performed, and the fourth sub-correction value is-4 ms; if 0-diff-t-s 2, quick broadcast is also performed, and the third sub-correction value is-2 ms.

And under the condition that the number difference is greater than zero and the number difference is less than zero, the processing of processing the correction value is carried out by equally dividing the time thresholds, so that the refined time control is realized, and the fluency of video playing is further ensured.

And S23, determining the current frame processing interval based on the sum of the current frame processing reference value and the current frame processing correction value.

Continuing with the above example, if Diff < = -2, play is delayed by 4ms based on the current frame processing reference value. Namely: the current frame processing interval is changed from 16.67ms to 16.67ms +4ms =20.67ms; if-2-straw diff-straw 0, slow playing is also executed, and playing is carried out after 2ms delay on the basis of the current frame processing reference value. Namely: the current frame processing interval is changed from 16.67ms to 16.67+2=18.67ms.

If Diff > =2, play is performed after 4ms reduction on the basis of the current frame processing reference value. Namely: the current frame processing interval is changed from 16.67ms to 16.67-4=12.67ms; and if 0-diff-straw 2, performing quick playing, and playing after reducing 2ms on the basis of processing the reference value by the current frame. Namely: the current frame processing interval is changed from 16.67ms to 16.67-2=14.67ms.

For the rest, please refer to S13 in the embodiment shown in fig. 5, which is not described herein again.

It should be noted that, when the target processing is decoding processing, the calculation method of the correction value processed by the current frame is the same as that of the correction value during the playing processing, and the difference is the specific values of the preset number, the first value and the second value that are specifically compared, so as to implement fast solution or slow solution of the decoding processing. These values are set according to actual requirements, and are not limited in any way.

And S24, extracting the current frame from the frame buffer area based on the current frame processing interval to perform target processing so as to play the processed video frame.

Specifically, the above S24 includes:

s241, acquiring the start time of the target process and the relative time of the previous frame.

The processing relative time is obtained by accumulating the frame processing interval from the first frame to the last frame of the target processing.

The starting time of the target processing is the time for starting the target processing, and the processing relative time is the result of frame processing interval accumulation. Specifically, the current processing is performed to the 10 th video frame, and then the processing relative time of the previous frame is the accumulated result of 9 frame processing intervals from the start of the first video frame to the 9 th video frame.

Further, in a case where the target process is a decoding process, acquiring a start time of the target process includes:

(1) And judging whether the frame numbers of the video frames are continuous or not.

(2) And when the frame numbers are not continuous, determining the time of the video frame with the discontinuous frame number appearing for the first time as the starting time of the target processing.

After receiving the video frame, analyzing the frame number FrameID of the video frame, and if the FrameID is discontinuous, determining a new target process, where a start time of the target process needs to be determined. Wherein the starting time of the target processing is the time when the video frame with discontinuous frame number appears for the first time.

For example, the frame number of the received video frame is 1,2,3,4, 100, 101, 102, the time when the video frame with the frame number of 100 is received is determined as the start time of the target process.

Whether the decoding process is performed newly is determined by the continuity of the frame numbers of the video frames, thereby simplifying the reliability of the determination of the start time.

S242, the current frame processing interval is accumulated to the processing relative time of the previous frame to obtain the processing relative time of the current frame.

After the current frame processing interval is obtained in S23, the sum of the current frame processing interval and the processing relative time of the previous frame is calculated to obtain the processing relative time of the current frame.

And S243, accumulating the starting time and the processing relative time of the current frame to obtain the target processing time of the current frame.

And then calculating the relative processing time of the starting time and the current frame to obtain the target processing time of the current frame.

And S244, if the current time reaches the target processing time of the current frame, extracting the current frame from the frame buffer area for target processing so as to play the processed video frame.

Comparing the current time with the target processing time of the current frame, and if the target processing time of the current frame is reached, extracting the current frame from a frame buffer area for target processing; and if the target processing time of the current frame is not reached, continuing to wait until the target processing time of the current frame is reached. The extracting of the current frame for the target processing is not particularly limited to directly processing the extracted current frame, and the extracting of the current frame and the adding of the current frame into the processing queue for waiting for processing may be performed.

In the video playing method provided by this embodiment, when the number difference is smaller than zero, it indicates that the number of video frames in the frame buffer area is small and slow processing is required, so that the first correction value larger than zero is used as the current frame processing correction value; when the number difference is greater than zero, it indicates that the number of video frames in the frame buffer is large and fast processing is required, and therefore, a second correction value smaller than zero is used as the current frame processing correction value. The slow processing or the fast processing is realized through the two conditions, so that the target processing time interval between any two adjacent frames is ensured to be within a controllable range. The starting time is a fixed time value, and the target processing time is judged by the accumulation of the starting time of the target processing and the processing relative time, so that the accuracy of the target processing time is ensured.

In some embodiments, the method further comprises:

(1) And judging whether the frame control time is abnormal or not, wherein the frame control time comprises at least one of the current time, the starting time and the processing relative time.

(2) When the frame control time is abnormal, the starting time and the processing relative time are reset.

When the time control is performed on the video frame in the frame buffer, it is also necessary to determine whether the frame control time is abnormal. If the current time is less than the starting time, the current time is abnormal; if the starting time is less than zero, the starting time is abnormal; if the processing relative time overflows, for example, the processing relative time exceeds 864000000, that is, if 10 days of continuous processing have been processed, it indicates that the processing relative time is abnormal. If any one of the above is abnormal, resetting the initial time and the processing time.

Under the condition that the frame control time is abnormal, the starting time and the processing relative time are reset, the target processing time is not influenced by the reset processing, and the reliability of the target processing is improved under the condition that the target processing is not influenced.

As a specific application example of the embodiment of the present invention, when the target processing is decoding, that is, before decoding, setting of a frame buffer is performed. Here, setting the frame buffer before decoding may be understood as smoothing control before decoding. As shown in fig. 7, the method includes:

(1) Start decoding for more than one second: since the decoding opposite terminals are various terminals, the specific video frame rate cannot be always known, and statistics needs to be performed during decoding. The statistical frame rate needs to be estimated for the first time at least 1 second later, so there is a "decoding start over 1 second" determination at the beginning of the process, i.e. frame rate statistics and frame rate correction are started after 1 second. When the frame rate cannot be counted within 1 second, the default frame rate is used, and the system sets the default frame rate to 60 frames.

(2) Frame rate statistics and correction: firstly, the frame rate is assumed to be 60 frames, namely, the system plays the frames according to the step length of 60 frames, and simultaneously recalculates the frame rate according to the number of the received frames after 1 second, and updates the frame rate into the smoothing control logic in real time.

(3) Whether to decode for the new time: the decision logic for the new decoding requires that the received frame numbers FrameID are consecutive. When a strong change in FrameID is detected, a new decode can be deemed, at which time the "decode first" flag and the "play first" flag are set.

(4) First decoding judgment: when decoding for the first time, two more steps need to be done. The first step is to judge the key frame, wait for the arrival of the key frame (IDR), if the non-key frame, need to abandon the current frame directly (the outer layer needs the logic of requesting the key frame), when the IDR frame arrives, have satisfied the precondition of decodable; and the second step is to judge whether the decoding buffer area reaches the reference quantity, if not, the received frame is only needed to be put into the buffer area to be decoded, and at the moment, the decoding cannot be carried out until the frame number in the buffer area to be decoded reaches the reference quantity, so that the jitter can be resisted through the fast-decoding and slow-decoding logic.

(5) Setting each mark: one is a "decoding start time" flag, which refers to the start time of the current decoding start, and the time is fixed during the continuous decoding process. When a new decoding is started, resetting the time by using the current time, wherein the function of the time is referred to decoding relative time; the second is a "decode relative time" flag, which is incremented by the time for each decoded frame relative to the decode start time. The time and the decoding start time are used together to determine whether the current frame should be decoded. The value accumulated each time the decoding relative time is composed of two parts ("time interval calculated from the frame rate" and "correction value calculated dynamically"); third, the "first decoding" flag is clear 0, which means that the early pre-buffering of decoding is completed and normal decoding cycle is started.

(6) Fast and slow solution logic: and calculating a correction value, wherein the correction value is obtained by combining empirical values of different scenes according to the number of video frames in a frame buffer area, the correction value needs to be adjusted and optimized according to actual conditions, and the specific method is similar to the smooth control logic before playing.

(7) Frame control time anomaly: the following situations can be determined as the frame control time is abnormal, and when the frame control time is abnormal, the decoding start time and the decoding relative time need to be reset to start a new round of frame control time control. a. The current time is less than the decoding start time; b. the decoding start time value is abnormal; c. the decoding relative time is prevented from overflowing, if the value of the decoding relative time exceeds 864000000, namely, the decoding is continuously performed for 10 days. The three cases described above are considered to be abnormal, resetting the values for two times does not have much impact on the smoothing control logic.

(8) The frame decoding time is up to: and detecting whether the current frame reaches the decoding time, if so, accumulating the decoding relative time and simultaneously informing subsequent logic to decode the frame, otherwise, continuously storing the frame into a queue to be decoded for decoding processing.

As a specific application example of the embodiment of the present invention, when the target process is playing, that is, before playing, setting of a frame buffer is performed. Here, setting the frame buffer before playing may also be understood as smooth control before playing. As shown in fig. 8, the smooth control logic before playing is similar to the smooth control logic before decoding, which is described above and is not repeated herein.

Fig. 4 shows a time interval (millisecond) between each frame of decoded data and the frame thereof after the post-decoding smoothing process is added. As can be seen from fig. 4, it becomes very smooth in the following, even in I-frame bursts.

However, in the first few tens of seconds of fig. 4, the frame rate is not stable, which is mainly caused by abrupt frame rate change due to frame rate statistics and inappropriate empirical value setting of the corresponding policy, which indicates that the convergence part in the previous stage needs to be optimized again, and the calculation logic of the correction value can be adjusted to achieve the optimization of convergence. Based on this, fig. 9 shows that the time interval (millisecond) between each frame of decoded data after convergence in the previous period and the previous frame is optimized, it can be seen that after the empirical parameter is adjusted according to the scene, the data can be converged quickly in about 1-2 seconds, the later is a relatively stable curve, and the time interval difference between two frames can be within 10ms, so that the purpose of smooth playing is achieved.

In this embodiment, a video playing apparatus is further provided, and the apparatus is used to implement the foregoing embodiments and preferred embodiments, and the description already made is omitted. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.

The present embodiment provides a video playback apparatus, as shown in fig. 10, including:

a first obtaining module 41, configured to obtain a current frame rate to determine a current frame processing reference value;

a second obtaining module 42, configured to obtain the number of video frames in a frame buffer, and determine a current frame processing correction value based on a size relationship between the number of video frames and a preset number;

a determining module 43, configured to determine a current frame processing interval based on a sum of the current frame processing reference value and the current frame processing correction value;

and the processing module 44 is configured to extract a current frame from the frame buffer based on the current frame processing interval to perform target processing, so as to play the processed video frame, where the target processing includes decoding processing and/or playing processing.

In some embodiments, the determining module 43 comprises:

the calculating unit is used for calculating the difference value between the video frame and the preset number and determining the number difference;

a first determining unit configured to determine a first correction value larger than zero as the current frame processing correction value when the number difference is smaller than zero;

a second determination unit configured to determine a second correction value smaller than zero as the current frame processing correction value when the number difference is larger than zero.

In some embodiments, the first determining unit includes:

a first determining subunit configured to determine, as the current frame processing correction value, a first sub-correction value that is greater than zero when the number difference is greater than a first value and less than zero, and determine, as the current frame processing correction value, a second sub-correction value that is greater than the first sub-correction value when the number difference is less than or equal to the first value;

and/or the presence of a gas in the gas,

the second determination unit includes:

a second determining subunit, configured to determine, as the current frame processing correction value, a third sub-correction value smaller than zero when the number difference is greater than zero and smaller than a second value, and determine, as the current frame processing correction value, a fourth sub-correction value smaller than the third sub-correction value when the number difference is greater than or equal to the second value.

In some embodiments, the processing module comprises:

an obtaining unit, configured to obtain a start time of the target processing and a processing relative time of a previous frame, where the processing relative time is obtained by accumulating frame processing intervals from a first frame of the target processing to the previous frame;

the first accumulation unit is used for accumulating the processing interval of the current frame to the processing relative time of the previous frame to obtain the processing relative time of the current frame;

the second accumulation unit is used for accumulating the starting time and the processing relative time of the current frame to obtain the target processing time of the current frame;

and the extracting unit is used for extracting the current frame from the frame buffer area to perform target processing when the current time reaches the target processing time of the current frame so as to play the processed video frame.

In some embodiments, in a case where the target process is a decoding process, the acquisition unit includes:

a judging subunit, configured to judge whether frame numbers of the video frames are consecutive;

and a third determining subunit, configured to determine, when the frame number is discontinuous, a time at which a video frame with a discontinuous frame number appears for the first time as a start time of the target process.

In some embodiments, the apparatus further comprises:

the judging module is used for judging whether frame control time is abnormal or not, wherein the frame control time comprises at least one of the current time, the starting time and the processing relative time;

and the resetting module is used for resetting the starting time and the processing relative time when the frame control time is abnormal.

In some embodiments, in a case where the target process is a decoding process, the first obtaining module includes:

a judging unit for judging whether the time for starting decoding exceeds a preset time;

and a third determining unit which determines a preset frame rate as the current frame rate when the time for starting decoding does not exceed the preset time.

The video playback device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and/or other devices that can provide the above-described functionality.

Further functional descriptions of the modules are the same as those of the corresponding embodiments, and are not repeated herein.

An embodiment of the present invention further provides an electronic device, which has the video playing apparatus shown in fig. 10.

Referring to fig. 11, fig. 11 is a schematic structural diagram of an electronic device according to an alternative embodiment of the present invention, and as shown in fig. 11, the electronic device may include: at least one processor 51, such as a CPU (Central Processing Unit), at least one communication interface 53, memory 54, at least one communication bus 52. Wherein a communication bus 52 is used to enable the connection communication between these components. The communication interface 53 may include a Display (Display) and a Keyboard (Keyboard), and the optional communication interface 53 may also include a standard wired interface and a standard wireless interface. The Memory 54 may be a high-speed RAM Memory (volatile Random Access Memory) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The memory 54 may alternatively be at least one memory device located remotely from the processor 51. Wherein the processor 51 may be in connection with the apparatus described in fig. 10, the memory 54 stores an application program, and the processor 51 calls the program code stored in the memory 54 for performing any of the above-mentioned method steps.

The communication bus 52 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus 52 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 11, but this is not intended to represent only one bus or type of bus.

The memory 54 may include a volatile memory (RAM), such as a random-access memory (RAM); the memory may also include a non-volatile memory (english: flash memory), such as a Hard Disk Drive (HDD) or a solid-state drive (SSD); the memory 54 may also comprise a combination of the above types of memories.

The processor 51 may be a Central Processing Unit (CPU), a Network Processor (NP), or a combination of a CPU and an NP.

The processor 51 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a Programmable Logic Device (PLD), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a field-programmable gate array (FPGA), a General Array Logic (GAL), or any combination thereof.

Optionally, the memory 54 is also used to store program instructions. The processor 51 may call program instructions to implement a video playing method as shown in any of the embodiments of the present application.

An embodiment of the present invention further provides a non-transitory computer storage medium, where a computer-executable instruction is stored in the computer storage medium, and the computer-executable instruction can execute the video playing method in any of the above method embodiments. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.

Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A video playback method, comprising:

2. The method of claim 1, wherein determining the current frame processing correction value based on a magnitude relationship between the number of video frames and a preset number comprises:

3. The method of claim 2, wherein determining a first correction value greater than zero as the current frame processing correction value when the difference is less than zero comprises:

and/or the presence of a gas in the atmosphere,

when the number difference is greater than zero, determining a first correction value greater than zero as the current frame processing correction value, including:

4. The method of claim 1, wherein extracting a current frame from the frame buffer for target processing based on the current frame processing interval to play a processed video frame comprises:

and if the current time reaches the target processing time of the current frame, extracting the current frame from the frame buffer area for target processing so as to play the processed video frame.

5. The method according to claim 4, wherein in the case that the target process is a decoding process, the obtaining a start time of the target process comprises:

judging whether the frame numbers of the video frames are continuous or not;

6. The method of claim 4, further comprising:

7. The method according to any of claims 1-6, wherein in case the target process is a decoding process, the obtaining the current frame rate comprises:

judging whether the time for starting decoding exceeds the preset time or not;

8. A video playback apparatus, comprising:

9. An electronic device, comprising:

a memory and a processor, the memory and the processor being communicatively connected to each other, the memory having stored therein computer instructions, the processor executing the computer instructions to perform the video playback method of any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions for causing a computer to execute the video playback method according to any one of claims 1 to 7.