CN116886924A - Stable frame rate self-driving method and device based on coded video - Google Patents
Stable frame rate self-driving method and device based on coded video Download PDFInfo
- Publication number
- CN116886924A CN116886924A CN202310988915.7A CN202310988915A CN116886924A CN 116886924 A CN116886924 A CN 116886924A CN 202310988915 A CN202310988915 A CN 202310988915A CN 116886924 A CN116886924 A CN 116886924A
- Authority
- CN
- China
- Prior art keywords
- frame rate
- video data
- video
- trigger
- triggers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000006641 stabilisation Effects 0.000 claims abstract description 37
- 238000011105 stabilization Methods 0.000 claims abstract description 37
- 230000008569 process Effects 0.000 abstract description 9
- 230000000087 stabilizing effect Effects 0.000 abstract description 2
- 239000000872 buffer Substances 0.000 description 9
- 230000008859 change Effects 0.000 description 9
- 230000036316 preload Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention provides a video stabilization frame rate self-driving method and a device, wherein the method comprises the following steps: requesting video data from a video data source; receiving video data from a video data source at a particular frame rate, the frame rate being n frames/second, where n is an integer; loading the video data with the appointed frame rate into an external cache video frame queue; setting triggering times m per second, and driving m triggering times per second, wherein m is an integer greater than n; setting n times of m triggers as effective triggers and m-n times as ineffective triggers; when the trigger is effective trigger, taking frames from the external cache video frame queue and outputting the frames; when the trigger is an invalid trigger, the trigger is ignored. The method and the device for stabilizing the frame rate self-driving based on the coded video not only can solve the problem that errors possibly exist in the process of driving and playing the video according to the frame interval, but also can adapt to the condition of changing the frame rate of video source data.
Description
Technical Field
The invention relates to the technical field of video communication, in particular to a stable frame rate self-driving method and device based on coded video.
Background
Currently, the video communication technology of the internet is widely applied to various video playing and streaming applications, which play an increasingly important role in our lives and works. For example, in video conferences used in work, various videos and live broadcast watched in life and the like are applications of the video broadcasting technology in real life, and the progress of the video broadcasting technology plays an important role in improving experience of watching live broadcast, participating in video conferences and the like.
Under the prior art, for example, in the video transmission process of a video conference, a sender collects video data according to a predetermined frame rate and transmits the video data to a receiver for playing, the receiver buffers the received video data in an external buffer video frame queue, and then takes the video data from the external buffer video frame queue according to a predetermined frame interval for decoding and playing. But this may cause errors due to inaccurate time of the frame interval, thereby affecting the actual playing effect of the video. For example, video data is fetched from an external cache video frame queue according to a frame rate of 30 frames for 1 second based on the frame rate of a sender, and in general, the frame intervals are all integer numbers, so if the frame interval time is 33 milliseconds at this time, an error of 10 milliseconds will be generated within one second, that is, the speed of fetching frames is slightly faster than the speed of receiving video frames, the error will accumulate to be bigger and bigger along with the increase of the playing duration of the video data, and the external cache video frame queue will be shorter and shorter until the external cache video frame queue is completely emptied and cannot fetch data along with the increase of the error, which will cause a clamping effect to affect the actual playing effect of video and reduce the experience of users. If the frame interval time is 34 ms at this time, an error of 20 ms is generated within one second, that is, the frame taking speed is slightly slower than the video frame receiving speed, the error is accumulated to be larger and larger along with the increase of the video data playing time length, and the external cache video frame queue is longer and longer along with the increase of the error until the predetermined queue length threshold is exceeded, and the effective video frame has to be discarded, so that the effect of actually playing the video is affected by the blocking, and the experience of a user is reduced.
In addition, if the video frame data is taken from the external buffer video frame queue by driving according to the frame interval, the time interval for taking the video frame data from the external buffer video frame queue is set to be a fixed time, and if the data frame rate of the video source changes due to the severe change of the video scene, the existing driving mode at this time cannot adapt to the change of the queue length caused by the network jitter or the changed frame rate of the video source, so that accidents such as video jam, picture distortion or play delay may occur.
Furthermore, in the prior art, if the frame rate required by the receiving side is relatively fixed in a frame taking manner at fixed time intervals, the video source may not have the data of the frame rate required by the user, and at this time, the video source is required to re-encode the original video data to generate the video data of the frame rate required by the user. Thus, not only a large amount of processor and memory resources are occupied, but also a certain time is required in the process of decoding and encoding, and the video data processing efficiency is affected.
Disclosure of Invention
In order to solve the problems that in the prior art, when the encoded video is processed, video frame data is taken from an external cache video frame queue according to frame rate driving, errors exist, and the frame rate cannot be adapted to change, the invention provides a stable frame rate self-driving method and device based on the encoded video, which not only can solve the problem that errors possibly exist in the process of driving and playing the video according to fixed frame intervals, but also can adapt to the situation that the frame rate of video source data changes.
In a first aspect, the present invention provides a video stabilization frame rate self-driving method, which is characterized in that the method includes:
requesting video data from a video data source;
receiving video data from a video data source at a particular frame rate, the frame rate being n frames/second, where n is an integer;
loading the video data with the appointed frame rate into an external cache video frame queue;
setting the triggering times per second as m, and driving m triggering times per second, wherein m is an integer greater than n;
setting n times of m triggers as effective triggers and m-n times as ineffective triggers;
when the trigger is effective trigger, taking frames from the external cache video frame queue and outputting the frames;
when the trigger is an invalid trigger, the trigger is ignored.
In a second aspect, the present invention further provides a video stabilization frame rate self-driving apparatus, which is characterized by comprising:
a request module for requesting video data from a video data source;
a receiving module, configured to receive video data from a video data source at a specific frame rate, where the frame rate is n frames/second, and n is an integer;
the loading module is used for loading the video data with the appointed frame rate into an external cache video frame queue;
the setting module is used for setting the triggering times per second to be m, driving m triggers per second, wherein m is an integer larger than n, setting n triggers in the m triggers to be effective triggers, and m-n triggers to be ineffective triggers;
and the judging module is used for taking frames from the external cache video frame queue and outputting the frames when the trigger is effective, and ignoring the trigger when the trigger is ineffective.
The invention provides a stable frame rate self-driving method and a device thereof: firstly, setting the driving times larger than the frame rate according to the frame rate of the video source data, so that the problem of video playing blocking caused by inaccurate triggering driving time due to triggering driving at fixed frame intervals can be avoided; secondly, the number of effective driving times is set according to the size of the video source data frame rate, and the effective driving times can be changed according to the change of the video source data frame rate, so that the method can adapt to the situation that the video source data frame rate changes due to video scene change or other reasons; thirdly, the frame rate of the data source video is adjusted according to the requirement of the user, so that the situation that the video source data need to be recoded to generate the video data with the frame rate required by the user because the video source does not have the data with the frame rate required by the user can be avoided, video delay can be reduced, and video playing efficiency can be improved; fourth, the frame fetching speed can be dynamically adjusted according to network fluctuation, so as to avoid data loss.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of a video stabilization frame rate self-driving method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a video stabilization frame rate self-driving device according to an embodiment of the present invention;
fig. 3A-3C are flowcharts of a video stabilization frame rate self-driving method according to an embodiment of the present invention.
Detailed Description
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Summary of The Invention
As described above, the present invention provides a method and apparatus for stabilizing frame rate self-driving based on encoded video, which not only can solve the problem that errors may exist in the process of driving and playing video according to frame intervals, but also can adapt to the situation of changing the frame rate of video source data, reduce the situation that video has a jam and delay in the playing process, and improve the experience of users watching video.
Exemplary method
Fig. 1 is a schematic diagram of a video stabilization frame rate self-driving method provided by an embodiment of the present invention, and fig. 3A to 3C are flowcharts of a video stabilization frame rate self-driving method provided by an embodiment of the present invention, and a preferred embodiment of the present invention is described below with reference to fig. 1 and fig. 3A to 3C, and the embodiment includes the following steps:
step 301: the video receiver 102 requests video data from the video data source 101;
step 302: the video receiver 102 receives the video data from the video data source 101 at a particular frame rate, the frame rate being n frames/second, where n is an integer; for example, the video source data frame rate is 30 frames/second.
Wherein the requested video frame rate is consistent with the video source data frame rate.
And when the frame rate of the video source data changes, the value of the implementation adjustment n is consistent with the frame rate.
For example, when the user views the video, the frame rate of the video source data is 30 frames/second, but the frame rate of the video becomes 40 frames/second due to the sudden occurrence of a scene with a severe change in the video or the switching of the video of the sender, at this time, the receiver adjusts the effective driving times to 40 times per second according to the changed frame rate of the video in real time, so that the situation of frame loss of the viewed video can be avoided. When the frame rate is changed, the sender will notify the receiver of the frame rate change in advance.
Step 303: loading the video data with the specified frame rate into an external cache video frame queue 103, and pre-loading the video data into the external cache video frame queue can effectively avoid the speed mutation of the received video frames caused by network jitter, so that the video playing is smooth and does not jam;
step 304, setting the triggering times per second as m, and driving m triggering times per second, wherein m is an integer greater than n; setting n times of m triggers as effective triggers and m-n times as ineffective triggers. The valid trigger indicates that a video frame can be extracted from the video frame queue, and the invalid trigger indicates that the extraction of the video frame from the video frame queue is suspended until the next valid trigger. The method has the advantages that the effective triggering times per second can be flexibly adjusted according to the frame rate of the received video frames, so that the speed of extracting the video frames from the video frame queue is completely consistent with the frame rate, and the situation that the queue is emptied or infinitely lengthened due to too fast or too slow extraction of the video frames is avoided.
Wherein n effective triggers are uniformly distributed in m triggers to ensure uniformity and smoothness of the frame rate.
Step 305: judging whether the current trigger is effective trigger, if yes, taking video frame data from an external cache video frame queue, judging whether the video frame data can be taken or not in step 306, and if yes, proceeding to step 307; if it is determined that video frame data is not fetched in the externally buffered video frame queue, then proceed to step 309. If it is determined that the current trigger is an invalid trigger, then proceed to step 312.
Step 307: when the trigger is a valid trigger and video data is fetched, the driving device 106 fetches frames from the external buffer video frame queue 103 and outputs them.
Step 308 is also performed on the basis of step 307;
step 308: when the data in the external buffer video frame queue 104 exceeds a predetermined first threshold, a subsequent predetermined number of invalid triggers are set as valid triggers, so as to achieve the purpose of accelerating frame fetching. When the data in the external buffer video frame queue exceeds a predetermined second threshold, wherein the second threshold is greater than the first threshold, discarding the predetermined frame number after n frames per second are taken out until the data in the external buffer video frame queue is lower than the predetermined second threshold, and then driving the next trigger to go to step 305 to continue to determine whether the next trigger is a valid trigger
For example, in the live broadcast watching process, when the data of the video source cannot be received at a certain moment due to the jitter of the network, a part of effective triggers are changed into ineffective triggers, meanwhile, the following ineffective triggers are changed into effective triggers, when the data exceeding a certain limit is suddenly received at the next moment, the data of an external cache video frame queue is increased, and the frame taking is accelerated due to the fact that the following effective triggers are denser, but when the data in the external cache video frame queue exceeds a preset first threshold value, a part of following ineffective triggers can be changed into effective triggers, and the situation that a part of effective triggers are changed into ineffective triggers due to the fact that the video source data cannot be received due to the network jitter before is further compensated, and the purpose is to ensure that the effective triggers are preset in number within one second, so that the loss of the data caused by the network jitter and other reasons is avoided. When the network jitter time is too long, the situation that the data of the external cache video frame queue 103 is too much can not be dealt with by the acceleration of frame fetching, namely when the data of the external cache video frame queue 103 exceeds a preset second threshold value, wherein the second threshold value is larger than the first threshold value, the situation that the data of the external cache video frame queue 103 excessively overflows in a mode of discarding a few frames before a key frame is avoided, and when the data of the external cache video frame queue is lower than the preset second threshold value, the frame is not lost any more. Since the part exceeding the second threshold is not discarded at one time here, but several frames before the key frame are discarded step by step in a plurality of steps, the video signal is prevented from being significantly jammed.
Care needs to be taken in the event of a frame loss to preserve the key frame, e.g., several frames before the key frame may be discarded to reduce the data in the external buffered video frame queue 103. Since frames following a key frame are encoded based on the key frame, the decoding of the following frames is not normally affected if the key frame is preserved.
Step 309: judging whether the continuous invalid triggering times are larger than or equal to a third threshold value, and if the continuous valid triggering is changed into invalid triggering times which are larger than or equal to the third threshold value, proceeding to step 310; if the number of consecutive valid triggers becomes invalid triggers is less than the third threshold, proceed to step 311.
Judging whether the number of times of changing continuous effective triggering into ineffective triggering is larger than a preset third threshold value or not; if it is determined whether the number of consecutive invalidation triggers is greater than the preset third threshold value, which indicates that the network may be interrupted or indeed is so bad that after the data in the external cache video frame queue 103 is emptied, all consecutive several valid triggers are changed to invalidation triggers to form consecutive invalidation triggers because the data is not fetched, in order to avoid that the black screen causes bad user experience to the user, the current video is temporarily switched to an alternative video stream, which is pre-stored in a pre-loaded video frame queue 104, and the flow proceeds to step 310; if the number of consecutive invalid triggers is less than the third threshold, then proceed to step 311.
Step 310: when the number of times that the continuous valid trigger becomes invalid trigger is greater than the preset third threshold, data is fetched from the preloaded video frame queue 104 for playing, and then the next trigger is driven, and the process proceeds to step 305, and whether the next trigger is a valid trigger is continuously determined. The alternative video stream stored in the preloaded video frame queue 104 may be a still picture such as a landscape background pattern, a conference room background pattern, a snap shot freeze scene during a conference, or a previously cached PPT page.
Wherein the frame rates of the pre-load video frame queue 104 and the external cache video frame queue 103 are consistent, or there are a plurality of pre-load video frame queues 104 with different frame rates, wherein at least one pre-load video frame queue 104 is consistent with the frame rate of the external cache video frame queue 103, so that when the drive cannot obtain data due to abnormal conditions, the data can be obtained from the pre-load video frame queues 104 with the same frame rate.
Assuming that the third threshold is set to k to 20, when k consecutive valid triggers are changed from invalid triggers to no data is fetched from the external cache video frame queue 103, the next valid trigger is to fetch data from the pre-load video frame queue 104 until the external cache video frame queue 103 has data, and then switch to fetch data from the external cache video frame queue 103.
For example, in the live broadcast watching process, due to abnormal conditions such as network delay, data of a video source is not received in time, at this time, the valid trigger cannot fetch the data from the external cache video frame queue 103, and if the valid trigger is specified to be valid for 20 times continuously, the data is not fetched from the external cache video frame queue 103 and becomes invalid, the next valid trigger fetches the data from the preloaded video frame queue 104. If one of the 20 continuous effective triggers can take data from the external cache video frame queue 103, the data is preferentially taken from the external cache video frame queue 103, and if the 20 continuous effective triggers have not taken data from the external cache video frame queue 103 and become invalid triggers, the next effective trigger takes data from the pre-loaded video frame queue 104, so that smoothness in the live broadcasting watching process can be ensured as much as possible.
The pre-load video frame queue 104 is consistent with the frame rate of the external cache video frame queue 103.
Step 311: when the number of times that the continuous effective trigger is changed into the ineffective trigger is smaller than a third threshold value and the current effective trigger does not acquire data from the external cache video frame queue, setting the effective trigger as the ineffective trigger, and setting a certain ineffective trigger after the effective trigger as the effective trigger at the same time, so that the effective trigger number of times within one second is ensured to be a preset number. The situation that no data is fetched from the external cache video frame queue is generally that the data transmission speed is reduced due to jitter of the network, so that the video frames in the external cache video frame queue are emptied, and the video frames are not fetched when one effective trigger is performed. In order to ensure the fixed frame taking times per second, the frame taking speed can be adaptively slowed down, and the frame taking times to be taken are shifted backwards. That is, if the video frame is not taken, the current valid trigger is set as the valid trigger, but a subsequent invalid trigger is changed to the valid trigger, for example, a subsequent 20 th invalid trigger may be set as the valid trigger. When the network is restored, video frame data can quickly enter an external cache video frame queue, and the effective trigger times at the back are denser due to the arrangement, so that the frame taking rate can be increased, and the overflow of the external cache video frame queue is avoided.
For example, when data with a required frame rate of 30 frames/second is requested from the data source, the driving trigger number may be set to 50 times per second, where 30 times are valid triggers, 20 times are invalid triggers, and 30 times are uniformly distributed among the 50 triggers, for example, the first time is valid trigger, the second time is invalid trigger, the third time is valid trigger, the fourth time is invalid trigger … …, and so on. If the effective trigger does not get the data due to network jitter and the like, the effective trigger is changed into the ineffective trigger, and the same number of ineffective triggers at the back are changed into the effective trigger at the same time, so as to ensure that the effective trigger times in one second are the preset number according to the frame rate of the sender, and avoid the loss of the data due to the network jitter and the like.
Step 312 is also performed after step 311;
step 312: when the trigger is an invalid trigger, the waiting device 107 keeps a waiting state, does not operate, drives the next trigger, proceeds to step 305, continues to determine whether the next trigger is a valid trigger, and does not fetch and output frames from the external cache video frame queue 103 until the next valid trigger.
For example, when the frame rate of the video source data is 30 frames/second, the number of triggers per second is set to be 50, wherein 30 triggers are valid triggers, 20 triggers are invalid triggers, wherein 30 valid triggers are uniformly distributed among the 50 triggers, when the triggers are valid triggers, frames are taken from an external cache queue and output, and otherwise, the triggers are ignored.
Since the original video data of the sender may not coincide with the frame rate required by the receiver, and since there are many receivers in the video conference, the frame rate required by each party is different, in order to reduce the encoding and decoding work required by the sender to perform video conversion, the sender may simply send the original video data, and the frame rate conversion performed by each receiver may change the frame rate by inserting frames and dropping frames to adapt to its own needs.
The embodiment of fig. 1 further includes a method for changing a frame rate by dropping frames shown in fig. 3B, which includes the steps of:
step 401: the video data source 101 is requested for video data at a specified frame rate.
Step 402: if the specified frame rate is less than the frame rate of the original encoded video data, the data is received at the frame rate of the original encoded video data.
Step 403: and discarding redundant data according to the specified frame rate and outputting video data with the specified frame rate.
For example, the specified frame rate is 25 frames/second, but the frame rate of the original encoded video is 30 frames/second, at which time the number of drive triggers per second is set to 70, with 30 active triggers, 40 inactive triggers, with 30 active triggers evenly distributed among the 70 triggers, with the inactive triggers being ignored directly, the active triggers not being output directly, but rather the frames preceding the key frames being selectively discarded until 25 frames of data are output within one second. By the method, the high-frame-rate video data can be directly converted into the low-frame-rate video data at the receiving side, decoding and encoding conversion at the transmitting side are not needed, and only the original video data are needed to be transmitted. This can increase the efficiency of the sender and can cope with different requirements of different receivers for the frame rate.
The fig. 1 embodiment also includes the method of changing the frame rate by interpolation shown in fig. 3C, which includes the steps of:
step 501: requesting video data of a specified frame rate from a video data source 101;
step 502: if the specified frame rate is greater than the frame rate of the original encoded video data, receiving data at the frame rate of the original encoded video data;
step 503: video data of a specified frame rate is generated and output by means of interpolation on the basis of the original encoded video data at the specified frame rate.
For example, the designated frame rate is 60 frames/second, but the frame rate of the original encoded video is 30 frames/second, at this time, the driving trigger number per second is set to be 50, wherein 30 times are valid triggers, and 20 times are invalid triggers, wherein the valid triggers are uniformly distributed in the middle of the invalid triggers, wherein the invalid triggers are directly ignored, the valid triggers are not directly output, and the previous frame data copied by one frame is inserted between every two valid trigger data until 60 frames of data are output within one second. The scheme can still adapt to the situation that the receiver requires a high frame rate, and the sender also sends the video data with the original frame rate to obtain the data with the required frame rate by inserting frames by the receiver.
The frame inserting method comprises the following steps:
repeated frame insertion (Frame Duplication): this method repeats existing frames in the video to increase the frame rate. For example, if the video script is 24 frames per second, you can increase the frame rate to 30 frames per second by repeating the frames.
Average interpolation (Frame Interpolation): this approach increases the frame rate by creating a new frame between two adjacent frames. Thus, a new smooth transition frame can be added in the video, so that the animation is smoother. For example, if the video script is 24 frames per second, you can increase the frame rate to 30 frames per second by mean interpolation. Although the interpolation can change the frame rate without re-encoding the entire video, it has some potential problems. Increasing the frame rate may result in the video becoming smoother, but may also result in the video losing some of its original image detail. Therefore, when using the interpolation technique, it is necessary to trade off advantages and disadvantages according to the specific situation and to ensure that the final video quality meets the expectations.
Exemplary System
Correspondingly, the embodiment of the invention also provides a video stabilization frame rate self-driving device. Fig. 2 is a block diagram of a distributed conference control system according to an embodiment of the present invention, where, as shown in the drawing, an apparatus provided in this embodiment includes: a request module 201, a receiving module 202, a loading module 203, a setting module 204 and a judging module 205, wherein
The request module 201 is configured to request video data from a video data source;
the receiving module 202 is configured to receive the video data from a video data source at a specific frame rate, where n is an integer;
the loading module 203 is configured to load the video data with the specified frame rate into an external cache video frame queue;
the setting module 204 is configured to set a trigger number m per second, and drive m triggers per second, where m is an integer greater than n, and n of the m triggers are set as valid triggers, and m-n are invalid triggers;
the judging module 205 is configured to take frames from the external cache video frame queue and output the frames when the trigger is an effective trigger, and ignore the trigger when the trigger is an ineffective trigger.
The receiving module 202 is configured to: when the frame rate changes, the value of the implementation adjustment n is consistent with the frame rate.
The setting module 204 is configured to: setting n times of triggering in m times per second as effective triggering and m-n times per second as ineffective triggering according to a predetermined rule; when no data is fetched from the queue at a certain valid trigger, the next valid trigger is set as an invalid trigger, and a certain invalid trigger thereafter is set as a valid trigger.
The setting module 204 is further configured to: the n valid triggers are uniformly arranged in m triggers.
The judging module 205 is configured to: and when no data is fetched from the external cache video frame queue for k continuous effective triggers, fetching data from the pre-loaded video frame queue until the external cache video frame queue has data, switching to fetching data from the external cache video frame queue.
The judging module 205 is further configured to: and when the data in the external cache video frame queue exceeds a preset first threshold value, setting the subsequent invalid trigger as an effective trigger, and accelerating frame taking.
The judging module 205 is further configured to: and when the data in the external cache video frame queue exceeds a preset second threshold, wherein the second threshold is larger than the first threshold, discarding the preset frame number after taking n frames per second until the data in the external cache video frame queue is lower than the preset second threshold.
The preloaded video frame queues are consistent with the frame rate of the externally cached video frame queues.
The request module 201 is further configured to: requesting video data specifying a frame rate from a video data source; receiving original encoded video data from the video data source if the specified frame rate is less than the frame rate of the original encoded video data; receiving data at a frame rate of the original encoded video data; discarding redundant data according to a specified frame rate; and outputting the video data at the specified frame rate.
The request module 201 is further configured to: requesting video data of a specified frame rate from a video data source; receiving original encoded video data from the video data source at a maximum frame rate if the specified frame rate is greater than the frame rate of the original encoded video data; receiving data at a frame rate of the original encoded video data; inserting frames according to the specified frame rate; and outputting the video data at the specified frame rate.
It should be noted that although the operations of the video stabilization frame rate self-driving method of the present invention are depicted in the drawings in a particular order, this does not require or imply that the operations be performed in that particular order, or that all of the illustrated operations be performed, in order to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.
Furthermore, although several modules of a video stabilization frame rate self-driving apparatus are mentioned in the above detailed description, this partitioning is merely exemplary and not mandatory. Indeed, the features and functions of two or more modules described above may be embodied in one module in accordance with embodiments of the present invention. Conversely, the features and functions of one module described above may be further divided into a plurality of modules to be embodied.
While the spirit and principles of the present invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments nor does it imply that features of the various aspects are not useful in combination, nor are they useful in any combination, such as for convenience of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
In accordance with the above disclosure of the present invention, the present invention provides:
1. a video stabilization frame rate self-driving method, the method comprising:
requesting video data from a video data source;
receiving video data from a video data source at a particular frame rate, the frame rate being n frames/second, where n is an integer;
loading the video data with the appointed frame rate into an external cache video frame queue;
setting the triggering times per second as m, and driving m triggering times per second, wherein m is an integer greater than n;
setting n times of m triggers as effective triggers and m-n times as ineffective triggers;
when the trigger is effective trigger, taking frames from the external cache video frame queue and outputting the frames;
when the trigger is an invalid trigger, the trigger is ignored.
2. The video stabilization frame rate self-driving method according to claim 1, wherein the step of requesting video data from the video data source further comprises:
requesting video data of a specified frame rate from a video data source;
receiving original encoded video data from the video data source if the specified frame rate is less than the frame rate of the original encoded video data;
receiving data at a frame rate of the original encoded video data;
discarding redundant data according to a specified frame rate; and
the specified frame rate video data is output.
3. The video stabilization frame rate self-driving method according to claim 1, wherein the step of requesting video data from the video data source further comprises:
requesting video data of a specified frame rate from a video data source;
receiving original encoded video data from the video data source at a maximum frame rate if the specified frame rate is greater than the frame rate of the original encoded video data;
receiving data at a frame rate of the original encoded video data;
inserting frames according to the specified frame rate; and
the specified frame rate video data is output.
4. The video stabilization frame rate self-driving method according to claim 1, wherein the step of receiving the video data from a video data source at a specific frame rate comprises:
when the frame rate changes, the value of the implementation adjustment n is consistent with the frame rate.
5. The video stabilization frame rate self-driving method according to any one of claims 1 to 4, wherein the setting n of m triggers as valid triggers and the m-n triggers as invalid triggers specifically comprises:
setting n times of triggering in m times per second as effective triggering and m-n times per second as ineffective triggering according to a predetermined rule;
when no data is fetched from the queue at a certain valid trigger, the next valid trigger is set as an invalid trigger, and a certain invalid trigger thereafter is set as a valid trigger.
6. The video stabilization frame rate self-driving method according to any one of claims 1 to 5, wherein the setting n of m triggers as valid triggers and m-n triggers as invalid triggers further comprises:
the n valid triggers are uniformly arranged in m triggers.
7. The method according to any one of claims 1-6, wherein when the trigger is a valid trigger, the step of fetching and outputting frames from the external buffered video frame queue comprises the steps of:
and when no data is fetched from the external cache video frame queue for k continuous effective triggers, fetching data from the pre-loaded video frame queue until the external cache video frame queue has data, switching to fetching data from the external cache video frame queue.
8. The method according to any one of claims 1-7, wherein when the trigger is a valid trigger, the step of fetching and outputting frames from the external buffered video frame queue further comprises the steps of:
and when the data in the external cache video frame queue exceeds a preset first threshold value, setting the subsequent invalid trigger as an effective trigger, and accelerating frame taking.
9. The method according to any one of claims 1-8, wherein when the trigger is a valid trigger, the step of fetching and outputting frames from the external buffered video frame queue further comprises the steps of:
and when the data in the external cache video frame queue exceeds a preset second threshold, wherein the second threshold is larger than the first threshold, discarding the preset frame number after taking n frames per second until the data in the external cache video frame queue is lower than the preset second threshold.
10. The method according to claim 7, wherein the preloaded video frame queues are consistent with the frame rate of the externally buffered video frame queues.
11. A video stabilization frame rate self-driving apparatus, comprising:
a request module for requesting video data from a video data source;
a receiving module, configured to receive video data from a video data source at a specific frame rate, where the frame rate is n frames/second, and n is an integer;
the loading module is used for loading the video data with the appointed frame rate into an external cache video frame queue;
the setting module is used for setting the triggering times per second to be m, driving m triggers per second, wherein m is an integer larger than n, setting n triggers in the m triggers to be effective triggers, and m-n triggers to be ineffective triggers;
and the judging module is used for taking frames from the external cache video frame queue and outputting the frames when the trigger is effective, and ignoring the trigger when the trigger is ineffective.
12. The video stabilization frame rate self-driving method according to claim 11, wherein the request module is further configured to:
requesting video data of a specified frame rate from a video data source;
receiving original encoded video data from the video data source if the specified frame rate is less than the frame rate of the original encoded video data;
receiving data at a frame rate of the original encoded video data;
discarding redundant data according to a specified frame rate; and
the specified frame rate video data is output.
13. The video stabilization frame rate self-driving apparatus according to claim 11, wherein the request module is further configured to:
requesting video data of a specified frame rate from a video data source;
receiving original encoded video data from the video data source at a maximum frame rate if the specified frame rate is greater than the frame rate of the original encoded video data;
receiving data at a frame rate of the original encoded video data;
inserting frames according to the specified frame rate; and
the specified frame rate video data is output.
14. The video stabilization frame rate self-driving apparatus according to claim 11, wherein the receiving module is configured to:
when the frame rate changes, the value of the implementation adjustment n is consistent with the frame rate.
15. The video stabilization frame rate self-driving apparatus according to any one of claims 11 to 14, wherein the setting module is configured to:
setting n times of triggering in m times per second as effective triggering and m-n times per second as ineffective triggering according to a predetermined rule;
when no data is fetched from the queue at a certain valid trigger, the next valid trigger is set as an invalid trigger, and a certain invalid trigger thereafter is set as a valid trigger.
16. The video stabilization frame rate self-driving apparatus according to any one of claims 11 to 15, wherein the setting module is further configured to:
the n valid triggers are uniformly arranged in m triggers.
17. The video stabilization frame rate self-driving apparatus according to any one of claims 11 to 16, wherein the judging module is configured to:
and when no data is fetched from the external cache video frame queue for k continuous effective triggers, fetching data from the pre-loaded video frame queue until the external cache video frame queue has data, switching to fetching data from the external cache video frame queue.
18. The video stabilization frame rate self-driving apparatus according to any one of claims 11 to 17, wherein the judging module is further configured to:
and when the data in the external cache video frame queue exceeds a preset first threshold value, setting the subsequent invalid trigger as an effective trigger, and accelerating frame taking.
19. The video stabilization frame rate self-driving apparatus according to any one of claims 11 to 18, wherein the judging module is further configured to:
and when the data in the external cache video frame queue exceeds a preset second threshold, wherein the second threshold is larger than the first threshold, discarding the preset frame number after taking n frames per second until the data in the external cache video frame queue is lower than the preset second threshold.
20. The video stabilization frame rate self-driving apparatus according to claim 17, wherein the preloaded video frame queues are consistent with the frame rate of the externally buffered video frame queues.
Claims (10)
1. A video stabilization frame rate self-driving method, the method comprising:
requesting video data from a video data source;
receiving video data from a video data source at a particular frame rate, the frame rate being n frames/second, where n is an integer;
loading the video data with the appointed frame rate into an external cache video frame queue;
setting the triggering times per second as m, and driving m triggering times per second, wherein m is an integer greater than n;
setting n times of m triggers as effective triggers and m-n times as ineffective triggers;
when the trigger is effective trigger, taking frames from the external cache video frame queue and outputting the frames;
when the trigger is an invalid trigger, the trigger is ignored.
2. The video stabilization frame rate self-driving method according to claim 1, wherein the step of requesting video data from a video data source further comprises:
requesting video data of a specified frame rate from a video data source;
receiving original encoded video data from the video data source if the specified frame rate is less than the frame rate of the original encoded video data;
receiving data at a frame rate of the original encoded video data;
discarding redundant data according to a specified frame rate; and
the specified frame rate video data is output.
3. The video stabilization frame rate self-driving method according to claim 1, wherein the step of requesting video data from a video data source further comprises:
requesting video data of a specified frame rate from a video data source;
receiving original encoded video data from the video data source at a maximum frame rate if the specified frame rate is greater than the frame rate of the original encoded video data;
receiving data at a frame rate of the original encoded video data;
inserting frames according to the specified frame rate; and
the specified frame rate video data is output.
4. The video stabilization frame rate self-driving method according to claim 1, wherein the step of receiving the video data from a video data source at a specific frame rate comprises:
when the frame rate changes, the value of the implementation adjustment n is consistent with the frame rate.
5. The video stabilization frame rate self-driving method according to any one of claims 1 to 4, wherein the setting n of m triggers as valid triggers and the m-n triggers as invalid triggers comprises:
setting n times of triggering in m times per second as effective triggering and m-n times per second as ineffective triggering according to a predetermined rule;
when no data is fetched from the queue at a certain valid trigger, the next valid trigger is set as an invalid trigger, and a certain invalid trigger thereafter is set as a valid trigger.
6. A video stabilization frame rate self-driving apparatus, comprising:
a request module for requesting video data from a video data source;
a receiving module, configured to receive video data from a video data source at a specific frame rate, where the frame rate is n frames/second, and n is an integer;
the loading module is used for loading the video data with the appointed frame rate into an external cache video frame queue;
the setting module is used for setting the triggering times per second to be m, driving m triggers per second, wherein m is an integer larger than n, setting n triggers in the m triggers to be effective triggers, and m-n triggers to be ineffective triggers;
and the judging module is used for taking frames from the external cache video frame queue and outputting the frames when the trigger is effective, and ignoring the trigger when the trigger is ineffective.
7. The video stabilization frame rate self-driving method according to claim 6, wherein the request module is further configured to:
requesting video data of a specified frame rate from a video data source;
receiving original encoded video data from the video data source if the specified frame rate is less than the frame rate of the original encoded video data;
receiving data at a frame rate of the original encoded video data;
discarding redundant data according to a specified frame rate; and
the specified frame rate video data is output.
8. The video stabilization frame rate self-driving apparatus according to claim 6, wherein the request module is further configured to:
requesting video data of a specified frame rate from a video data source;
receiving original encoded video data from the video data source at a maximum frame rate if the specified frame rate is greater than the frame rate of the original encoded video data;
receiving data at a frame rate of the original encoded video data;
inserting frames according to the specified frame rate; and
the specified frame rate video data is output.
9. The video stabilization frame rate self-driving apparatus according to claim 6, wherein the receiving module is configured to:
when the frame rate changes, the value of the implementation adjustment n is consistent with the frame rate.
10. The video stabilization frame rate self-driving apparatus according to any one of claims 6 to 9, wherein the setting module is configured to:
setting n times of triggering in m times per second as effective triggering and m-n times per second as ineffective triggering according to a predetermined rule;
when no data is fetched from the queue at a certain valid trigger, the next valid trigger is set as an invalid trigger, and a certain invalid trigger thereafter is set as a valid trigger.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310988915.7A CN116886924A (en) | 2023-08-07 | 2023-08-07 | Stable frame rate self-driving method and device based on coded video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310988915.7A CN116886924A (en) | 2023-08-07 | 2023-08-07 | Stable frame rate self-driving method and device based on coded video |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116886924A true CN116886924A (en) | 2023-10-13 |
Family
ID=88262184
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310988915.7A Pending CN116886924A (en) | 2023-08-07 | 2023-08-07 | Stable frame rate self-driving method and device based on coded video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116886924A (en) |
-
2023
- 2023-08-07 CN CN202310988915.7A patent/CN116886924A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109660879B (en) | Live broadcast frame loss method, system, computer equipment and storage medium | |
| US7562375B2 (en) | Fast channel change | |
| EP2095205B1 (en) | Hybrid buffer management | |
| CN107566918B (en) | A kind of low delay under video distribution scene takes the neutrel extraction of root | |
| US8711929B2 (en) | Network-based dynamic encoding | |
| US10164736B2 (en) | Systems, methods, apparatus, and computer program products for providing forward error correction with low latency | |
| KR101330907B1 (en) | Method for reducing channel change times in a digital video apparatus | |
| KR100643270B1 (en) | Client and method for playing video stream | |
| US20180213296A1 (en) | Assisted acceleration for video streaming clients | |
| CN108965807B (en) | Real-time video playing control method and device, terminal and storage medium | |
| WO2018005083A1 (en) | Method and system for dynamically adjusting a camera setting for a video stream | |
| CN111385647B (en) | Method for controlling streaming buffer in media player and related buffering device | |
| CN112866746A (en) | Multi-path streaming cloud game control method, device, equipment and storage medium | |
| CN111866526B (en) | Live broadcast service processing method and device | |
| CN112423074B (en) | Audio and video synchronization processing method and device, electronic equipment and storage medium | |
| CN111866533B (en) | Live broadcast transcoding method and device | |
| CN113382278A (en) | Video pushing method and device, electronic equipment and readable storage medium | |
| CN116886924A (en) | Stable frame rate self-driving method and device based on coded video | |
| US6757332B1 (en) | Video decoding method, video decoding apparatus, and program storage media | |
| CN105306970B (en) | A kind of control method and device of live streaming media transmission speed | |
| CN115379235A (en) | Image decoding method and device based on buffer pool, readable medium and electronic equipment | |
| US6990144B2 (en) | System and method for overrun catch-up in a real-time software | |
| US20230059063A1 (en) | Method and device for adaptive playout of media content | |
| EP3337174B1 (en) | Method and device for transmission of content | |
| CN117793075A (en) | Network computer multimedia video playing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |