CN112468875A - Display output control method and device of video decoding frame, storage medium and terminal - Google Patents
Display output control method and device of video decoding frame, storage medium and terminal Download PDFInfo
- Publication number
- CN112468875A CN112468875A CN202011373449.4A CN202011373449A CN112468875A CN 112468875 A CN112468875 A CN 112468875A CN 202011373449 A CN202011373449 A CN 202011373449A CN 112468875 A CN112468875 A CN 112468875A
- Authority
- CN
- China
- Prior art keywords
- frame
- decoded
- frames
- display
- target frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 239000000872 buffer Substances 0.000 claims abstract description 55
- 238000004590 computer program Methods 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 7
- 230000003993 interaction Effects 0.000 description 7
- 230000007774 longterm Effects 0.000 description 7
- 238000002474 experimental method Methods 0.000 description 4
- 239000003550 marker Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 241000023320 Luma <angiosperm> Species 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000012536 storage buffer Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
A display output control method and device, storage medium and terminal of video decoding frame are disclosed, the method includes: when each frame of video data is decoded by a decoder, storing frame level information of each frame into a delay queue, and storing the decoded data of each frame into a frame buffer space of a display module; acquiring a frame with the most front display sequence as a target frame according to frame level information stored in the delay queue; judging whether the target frame is a reference frame or not, if so, detecting whether the number of frames in the delay queue is greater than a preset value or not, and copying the data of the target frame to a decoder frame buffer space when the number of frames in the delay queue is greater than the preset value; outputting the target frame for display; and repeatedly executing the steps until the video data is decoded. Therefore, the method can avoid copying each frame of data and overcome the problem that the decoded frame can not be output for a long time.
Description
Technical Field
The present invention relates to the field of video display, and in particular, to a method and an apparatus for controlling display output of a video decoded frame, a storage medium, and a terminal.
Background
Video playing mainly involves decoding and displaying two major parts, namely, video needs to be firstly decoded into original data by a decoder (usually in YUV format), and then rendered to a screen via a module responsible for image display (Graphics Processing Unit, GPU), wherein, YUV, which is a color coding method, "Y" represents brightness (Luma), i.e., a gray scale value, and "U" and "V" represent chrominance, video coding is a process of compression-coding original video data according to a certain standard (or protocol); video decoding is the process of restoring encoded video data, namely the inverse process of video coding, video frames are decoded by a decoder according to a decoding sequence; the video display is a process of rendering decoded video data to a screen, and the decoded video frames are displayed in real time in sequence.
However, in the conventional video decoding and displaying scheme, due to the data interaction logic of the decoder and the display module, a problem may occur that the amount of data of the copied video frame is too large to cause an inefficient decoding and displaying of the video, or a problem may occur that a frame buffer of the display module cannot be released for a long time due to too many reference frames or long-term reference frames, so that the video frame cannot be displayed for a long time. The inter-frame predicted video frames need to refer to already decoded video frames to complete decoding, and the frames used as references are reference frames.
Disclosure of Invention
The invention solves the technical problem of how to avoid copying each frame of data and solve the problem that the decoded frame can not be output for a long time due to excessive number of reference frames or the appearance of long-term reference frames.
To solve the above problem, an embodiment of the present invention provides a method for controlling display output of a video decoded frame, where the method includes: step A, when each frame of video data is decoded by a decoder, storing frame level information of each frame into a delay queue, and storing the decoded data of each frame into a frame buffer space of a display module; b, acquiring the frame with the most front display sequence as a target frame according to the frame level information stored in the delay queue; step C, judging whether the target frame is a reference frame, and if so, executing step D; step D, detecting whether the number of frames in the delay queue is greater than a preset value, and executing the step E when the number of frames in the delay queue is greater than the preset value; e, copying the data of the target frame into a frame buffer space of a decoder; step F, outputting the target frame for display; and repeating the steps A to F until the video data is decoded.
Optionally, step C further includes: and if the judgment result is negative, skipping to the step F, and outputting the target frame for display.
Optionally, step D further includes: and when the number of frames in the delay queue is less than or equal to the preset value, not outputting any frame for displaying.
Optionally, step B further includes: and C, detecting whether the target frame meets necessary conditions, and if so, continuing to execute the step C.
Optionally, after detecting whether the target frame meets the necessary condition, the method further includes: and if not, skipping to execute the step A.
Optionally, the requirements include: all frames of the video data that are not decoded are displayed in an order following the target frame.
Optionally, the method further includes: when each frame of video data is decoded by a decoder, determining whether each decoded frame is still a reference frame or not so as to mark each frame determined as the reference frame; the step C comprises the following steps: detecting whether a mark exists on the target frame; if the mark exists on the target frame, the judgment result of the step C is yes; and C, if the mark does not exist on the target frame, judging that the judgment result in the step C is negative.
An embodiment of the present invention further provides a device for controlling display output of a video decoded frame, where the device includes: the decoding module is used for executing the step A, storing the frame level information of each frame into the delay queue when the decoder decodes each frame of the video data, and storing the decoded data of each frame into the frame buffer space of the display module; a target frame obtaining module, configured to execute step B, obtain, according to the frame level information stored in the delay queue, a frame with the most advanced display order as a target frame; a reference frame judging module, configured to execute step C, judge whether the target frame is a reference frame, and if the judgment result is yes, execute step D; a queue detection module, configured to perform step D, detect whether a number of frames in the delay queue is greater than a preset value, and perform step E when the number of frames in the delay queue is greater than the preset value; a copy module, configured to perform step E, copy the data of the target frame into a decoder frame buffer space; a display module, configured to perform step F, and output the target frame for display; and a repeating module for repeating the steps A to F until the video data is decoded.
Embodiments of the present invention further provide a storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform any of the steps of the method.
The embodiment of the present invention further provides a terminal, which includes a memory and a processor, where the memory stores a computer program that can be executed on the processor, and the processor executes any one of the steps of the method when executing the computer program.
Compared with the prior art, the technical scheme of the embodiment of the invention has the following beneficial effects:
compared with the prior art, the display output control method of the video decoding frame provided by the embodiment of the invention can back up some reference frames in the decoder according to specific application occasions. The method has the advantages that the method avoids copying each frame of data, can adjust the maximum delay interval of the display frame according to the practical application scene, achieves balance between efficiency and delay, and further can solve the problem that the decoded frame cannot be output for a long time due to the fact that the number of reference frames is too large or long-term reference frames appear.
Further, the scheme of the embodiment of the present invention is common to all mainstream video coding protocols at present, and videos in experiments include h.264, h.265, VP8, VP9, MPEG4, and the like. According to the scheme, some reference frames can be backed up in the decoder according to specific application. Specifically, the more sensitive the application scenario is to the output delay, or the greater the number of reference frames used by a particular film source during encoding, the greater the probability of occurrence of backup reference frames.
Drawings
FIG. 1 is a diagram illustrating a first prior art data interaction between a decoder and a display module;
FIG. 2 is a diagram illustrating a second prior art decoder and display module for data interaction;
FIG. 3 is a flowchart illustrating a first method for controlling display output of decoded video frames according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating a second method for controlling display output of decoded video frames according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a display output control apparatus for video decoded frames according to an embodiment of the present invention.
Detailed Description
As mentioned in the background, in the conventional video decoding display scheme, there is a problem due to the data interaction logic between the decoder and the display module.
Specifically, there are two common methods for data interaction between the decoder and the display module: referring to fig. 1, fig. 1 is a schematic diagram of data interaction between a first decoder and a display module in the prior art, where the decoder and the display module respectively maintain a plurality of buffers, a buffer area maintained by the display module is an area 1,2,3,4, …, etc. in the diagram, and a buffer area maintained by the decoder is an area 1 ', 2', 3 ', 4', …, etc. in the diagram. Each time a frame of data is decoded, the system performs a data COPY (COPY), copies the data in the decoding buffer to the display buffer (as shown in fig. 1, a data COPY is performed for each region), and then sequentially displays the data in the display buffer in the display order.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating data interaction between a second decoder and a display module in the prior art. It is common practice that the decoder does not allocate a buffer therein, each decoded frame data is directly stored in the buffer (1,2,3,4, …, n) allocated by the display module, and when the display module displays the decoded data (i.e. the buffer 1 and the buffer n are sent to the display module), the corresponding frame buffer is released (e.g. after the buffer n is displayed in fig. 2, the buffer n is released). In the decoding process, since most of the frames are to be used as reference frames of the subsequent video frames, and there are frames displayed before (in decoding order) and after (in display order) the decoding exists, the decoded frames cannot be immediately output to the display module for display, but need to be retained for a period of time inside the decoder. In this scheme, a precondition that a frame can be output for display is that the frame is not or has not been used as a reference frame. This scheme has no data copy at all, compared to the scheme of fig. 1.
The first scheme in fig. 1 needs to copy each frame, which is inefficient, especially when the resolution of the video to be decoded is large, the time consumption for copying data occupies a considerable part of the whole decoding process, and the efficiency of the video decoding display is greatly affected. In the method of the second scheme in fig. 2, in which the display module and the decoder share the buffer, the decoded video frame may remain inside the decoder for a long time until it is marked as a non-reference frame and then can not be displayed. Taking the most common h.264 video coding protocol as an example, the protocol provides that at most 16 reference frames may exist simultaneously, which may not occur continuously, and may also exist long-term reference frames (long term reference frames), so that in the worst case, although tens of frames are already decoded, the frame which should be output and displayed first still serves as a reference frame, and thus no frame can be displayed. The inability to display video frames for long periods of time in the second scheme may be unacceptable in some situations.
To solve the above problem, an embodiment of the present invention provides a method for controlling display output of a video decoded frame, including: step A, when each frame of video data is decoded by a decoder, storing frame level information of each frame into a delay queue, and storing the decoded data of each frame into a frame buffer space of a display module; b, acquiring the frame with the most front display sequence as a target frame according to the frame level information stored in the delay queue; step C, judging whether the target frame is a reference frame, and if so, executing step D; step D, detecting whether the number of frames in the delay queue is greater than a preset value, and executing the step E when the number of frames in the delay queue is greater than the preset value; e, copying the data of the target frame into a frame buffer space of a decoder, and releasing a frame buffer space of a display module for storing the data of the target frame; step F, outputting the target frame for display; and repeating the steps A to F until the video data is decoded.
By the scheme, the problem that the decoded frame cannot be output for a long time due to the excessive number of reference frames or the occurrence of long-term reference frames is solved while the data of each frame is prevented from being copied.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
Referring to fig. 3, an embodiment of the present invention provides a method for controlling display output of a video decoded frame, where the method includes:
step A, when each frame of video data is decoded by a decoder, storing frame level information of each frame into a delay queue, and storing the decoded data of each frame into a frame buffer space of a display module;
and a buffer space is distributed in the display module and used for storing the data of each frame to be displayed, and the buffer space is recorded as a frame buffer space of the display module. When each frame is decoded by the decoder, a frame buffer is selected from the frame buffer space of the display module and is sent to the decoder, and the decoder stores the decoded data into the selected frame buffer.
The frame level information of each frame is various attributes or description information of the frame, and may include information such as a frame type, a display order of the frame, whether the frame is a reference frame, and an address of a display buffer.
A delay queue is defined at the decoder side, and is used for storing frame-level information of a plurality of frames decoded by the decoder, and is used for controlling to copy part of frames in the frame buffer space of the display module into the frame buffer space of the decoder, and the specific control mode is as follows. The decoder is used for decoding video data, and may be a hardware decoder or a software decoder, which is a hardware decoder in this embodiment; the delay queues (also called delay queues) can be implemented by RabbitMQ, DelayQueue, and the like.
B, acquiring the frame with the most front display sequence as a target frame according to the frame level information stored in the delay queue;
the delay queue stores frame level information of a plurality of frames, and frame data corresponding to the plurality of frames are stored in a frame buffer space of the display module.
And selecting the frame with the most front display sequence from the delay queue as a target frame each time, and judging whether the target frame needs to be copied before being displayed according to the following steps C to E.
Optionally, the frame level information includes a display order of the frames, and a frame with a top display order among the frames in the delay queue may be selected according to the frame level information.
Step C, judging whether the target frame is a reference frame, and if so, executing step D;
if the target frame is a reference frame, the frame to be displayed subsequently needs to be decoded by using the target frame as a reference frame, and may need to be copied.
Step D, detecting whether the number of frames in the delay queue is greater than a preset value, and executing the step E when the number of frames in the delay queue is greater than the preset value;
e, copying the data of the target frame into a frame buffer space of a decoder;
step F, outputting the target frame for display, and releasing a frame buffer space of a display module for storing the data of the target frame;
the preset value (Threshold) is a preset number of frames, and the preset value can be adjusted according to different application scenarios. For example, 5 frames, 10 frames, etc. may be set. When the number of frames corresponding to the frame level information contained in the delay queue (i.e., the number of frames in the delay queue) is greater than a preset value, the target frame needs to be output for display.
A storage buffer space exists in the decoder, the buffer space is used for storing data of a plurality of frames copied from the frame buffer space of the display module, and the buffer space is recorded as a frame buffer space of the decoder. When the target frame is a reference frame and the frame needs to be output and displayed, the data of the target frame stored in the frame buffer space of the display module is copied to the frame buffer space of the decoder to be used as a reference for decoding of a subsequent frame. The target frame can be directly output and displayed, and the frame buffer space of the display module for storing the data of the target frame is released after the target frame is displayed.
Through a large number of video playing experiments, it is known that when the preset value (Threshold) is 10 frames, all usage scenarios can be basically satisfied, that is, a decoder will display one frame after delaying for 10 frames at most. As calculated at a frame rate of 30, corresponds to a delay of about 333 ms. Also in experiments it was found that the number of frames that need to be copied per second is within 4 frames.
And repeating the steps A to F until the video data is decoded.
And D, finishing the output display of one frame through the steps B to F, and if the next frame needs to be displayed, executing the steps B to F again. And the decoder continuously decodes the video data, and stores the frame level information of each frame which is newly decoded into the delay queue until the decoding of the last frame of the video data is completed.
The scheme in fig. 1 is common to all mainstream video coding protocols at present, and videos in experiments include h.264, h.265, VP8, VP9, MPEG4 and the like. According to the scheme, some reference frames can be backed up in the decoder (namely, some reference frames are copied to the internal buffer of the decoder from the buffer of the display module) according to specific application occasions. Specifically, the more sensitive the application scenario is to the output delay, or the greater the number of reference frames used by a particular film source during encoding, the greater the probability of occurrence of backup reference frames.
Therefore, the method avoids copying each frame of data, can adjust the maximum delay interval of the display frame according to the practical application scene, and balances efficiency and delay, thereby overcoming the problem that the decoded frame can not be output for a long time due to excessive number of reference frames or long-term reference frames.
In one embodiment, referring to fig. 4, the step C in fig. 3 further includes: and if the judgment result is negative, skipping to the step F, and outputting the target frame for display.
And C, when the target frame is judged not to be the reference frame in the step C, directly outputting the target frame for display, and releasing the frame buffer space of the display module for storing the data of the target frame.
Optionally, step D further includes: and when the number of frames in the delay queue is less than or equal to the preset value, jumping to step S401, and not outputting any frame for displaying.
And if the number of frames in the delay queue is less than the preset value, the display frame is not output for the moment.
Optionally, after the video data is decoded, the delay queue may be controlled to output frames in the display delay queue according to the display order, so as to display each frame of the video data.
In one embodiment, please continue to refer to fig. 4, said step B further includes: step S402, detecting whether the target frame meets the necessary conditions, and if so, continuing to execute the step C.
Optionally, after detecting whether the target frame meets the requirement in step S402, the method further includes: and if not, skipping to execute the step A.
Optionally, the requirements include: all frames of the video data that are not decoded are displayed in an order following the target frame.
After each target frame is acquired, whether the target frame meets a necessary condition is further detected, that is, whether subsequent undecoded frames are behind the target frame is met, and if the target frame does not meet the condition, the decoding can be continued to acquire the target frame meeting the necessary condition.
In one embodiment, the video data is in h.264 format, and the display Order of each frame is identified by a specific sequence number, i.e. a value of video frame display Order (POC). There is one POC per frame, the display order being in the order of increasing POC, the POC of the first frame of a video sequence being equal to 0. If the POC of the target frame is 0 or the POC of the target frame is consecutive to the POC of a frame that has been displayed before, it can be concluded that the frames that have not been decoded after are all displayed after the target frame. POC will count again from 0 after increasing to a certain value.
It should be noted that there are slight differences between different encoding and decoding standards, and other encoding and decoding manners can be referred to the above h.264 or whether the target frame meets the necessary condition is detected by the existing method of each encoding and decoding standard.
In one embodiment, the method further comprises: when each frame of video data is decoded by a decoder, determining whether each decoded frame is still a reference frame or not so as to mark each frame determined as the reference frame; the step C comprises the following steps: detecting whether a mark exists on the target frame; if the mark exists on the target frame, the judgment result of the step C is yes; and C, if the mark does not exist on the target frame, judging that the judgment result in the step C is negative.
Wherein, during the decoding of each frame of video data by the decoder, some reference frames within the decoder may be re-marked as non-reference frames. That is, it is determined whether or not each frame marked as a reference frame is determined as a reference frame at the time of decoding, and marking is performed on a frame determined as a reference frame by a decoder; if the decoder determines that a certain frame is a non-reference frame, the frame is not marked; if it is preceded by a marker of the reference frame, the marker is deleted. Correspondingly, in step C, whether each frame is a reference frame may be detected according to whether the mark of the reference frame exists on each frame. The number of reference frames is typically determined by the protocol.
In contrast to the scheme of fig. 1, the scheme described by the embodiment of the present invention does not require frequent copying of data between the display buffer and the decode buffer. Compared with the scheme of fig. 2, the invention does not have the situation that a plurality of frames cannot be decoded and still one frame cannot be displayed.
Referring to fig. 5, an embodiment of the present invention further provides a display output control apparatus 50 for video decoded frames, including:
a decoding module 501, configured to execute step a, store frame-level information of each frame into a delay queue when each frame of video data is decoded by a decoder, and store the decoded data of each frame into a frame buffer space of a display module;
a target frame acquiring module 502, configured to execute step B, and acquire, according to the frame level information stored in the delay queue, a frame with the most advanced display order as a target frame;
a reference frame determining module 503, configured to perform step C, determine whether the target frame is a reference frame, and if the determination result is yes, perform step D;
a queue detection module 504, configured to perform step D, detect whether a number of frames in the delay queue is greater than a preset value, and perform step E when the number of frames in the delay queue is greater than the preset value;
a copy module 505, configured to perform step E, copying the data of the target frame into a decoder frame buffer space;
a display module 506, configured to perform step F, and output the target frame for display;
a repeating module 507, configured to repeatedly execute the steps a to F until the video data is decoded.
In an embodiment, the reference frame determining module 503 is further configured to jump to perform the step F and output the target frame for display when the determination result is negative.
In an embodiment, the queue detecting module 504 is further configured to not output any frame for displaying when the number of frames in the delay queue is less than or equal to the preset value.
In an embodiment, the target frame obtaining module 502 is further configured to detect whether the target frame meets a requirement, and if so, continue to execute the step C.
In one embodiment, the target frame acquiring module 502 is further configured to jump to perform the step a when the target frame does not satisfy the requirement after detecting whether the target frame satisfies the requirement.
Optionally, the requirements include: all frames of the video data that are not decoded are displayed in an order following the target frame.
In one embodiment, the apparatus 50 for controlling display output of the video decoded frame further comprises:
a reference frame determination module, configured to determine whether each decoded frame is still a reference frame when each frame of the video data is decoded by the decoder, so as to mark each frame determined as a reference frame;
the reference frame determining module 503 includes:
a mark detection unit for detecting whether a mark exists on the target frame;
a reference frame determining unit, configured to determine that the determination result in step C is yes if a mark exists on the target frame;
and C, a reference frame excluding unit, configured to determine that the determination result in the step C is negative if the target frame has no mark.
For more details of the operation principle and the operation mode of the display output control apparatus 50 for video decoded frames, reference may be made to the related description of the methods in fig. 3 to fig. 4, which is not repeated here.
An embodiment of the present invention further provides a storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the method in any one of fig. 3 to 4. The storage medium may be a computer-readable storage medium, and may include, for example, a non-volatile (non-volatile) or non-transitory (non-transitory) memory, and may further include an optical disc, a mechanical hard disk, a solid state hard disk, and the like.
An embodiment of the present invention further provides a terminal, where the terminal may include a memory and a processor, where the memory stores a computer program that is executable on the processor, and the processor executes the steps of the method in fig. 3 to 4 when executing the computer program.
Specifically, in the embodiment of the present invention, the processor may be a Central Processing Unit (CPU), and the processor may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
It will also be appreciated that the memory in the embodiments of the subject application can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash memory. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of example and not limitation, many forms of Random Access Memory (RAM) are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (enhanced SDRAM), SDRAM (SLDRAM), synchlink DRAM (SLDRAM), and direct bus RAM (DR RAM).
It should be understood that the term "and/or" herein is merely one type of association relationship that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in this document indicates that the former and latter related objects are in an "or" relationship.
The "plurality" appearing in the embodiments of the present application means two or more.
The descriptions of the first, second, etc. appearing in the embodiments of the present application are only for illustrating and differentiating the objects, and do not represent the order or the particular limitation of the number of the devices in the embodiments of the present application, and do not constitute any limitation to the embodiments of the present application.
The term "connect" in the embodiments of the present application refers to various connection manners, such as direct connection or indirect connection, to implement communication between devices, which is not limited in this embodiment of the present application.
Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A method for controlling display output of a video decoded frame, the method comprising:
step A, when each frame of video data is decoded by a decoder, storing frame level information of each frame into a delay queue, and storing the decoded data of each frame into a frame buffer space of a display module;
b, acquiring the frame with the most front display sequence as a target frame according to the frame level information stored in the delay queue;
step C, judging whether the target frame is a reference frame, and if so, executing step D; step D, detecting whether the number of frames in the delay queue is greater than a preset value, and executing the step E when the number of frames in the delay queue is greater than the preset value;
e, copying the data of the target frame into a frame buffer space of a decoder;
step F, outputting the target frame for display;
and repeating the steps A to F until the video data is decoded.
2. The method of claim 1, wherein step C further comprises:
and if the judgment result is negative, skipping to the step F, and outputting the target frame for display.
3. The method of claim 1, wherein step D further comprises:
and when the number of frames in the delay queue is less than or equal to the preset value, not outputting any frame for displaying.
4. The method of claim 1, wherein step B further comprises:
and C, detecting whether the target frame meets necessary conditions, and if so, continuing to execute the step C.
5. The method of claim 4, wherein after detecting whether the target frame satisfies a requirement, the method further comprises:
and if not, skipping to execute the step A.
6. The method according to claim 4 or 5, wherein the requirements comprise:
all frames of the video data that are not decoded are displayed in an order following the target frame.
7. The method of any of claims 1 to 5, further comprising:
when each frame of video data is decoded by a decoder, determining whether each decoded frame is still a reference frame or not so as to mark each frame determined as the reference frame;
the step C comprises the following steps:
detecting whether a mark exists on the target frame;
if the mark exists on the target frame, the judgment result of the step C is yes;
and C, if the mark does not exist on the target frame, judging that the judgment result in the step C is negative.
8. An apparatus for controlling display output of a video-decoded frame, the apparatus comprising:
the decoding module is used for executing the step A, storing the frame level information of each frame into the delay queue when the decoder decodes each frame of the video data, and storing the decoded data of each frame into the frame buffer space of the display module;
a target frame obtaining module, configured to execute step B, obtain, according to the frame level information stored in the delay queue, a frame with the most advanced display order as a target frame;
a reference frame judging module, configured to execute step C, judge whether the target frame is a reference frame, and if the judgment result is yes, execute step D;
a queue detection module, configured to perform step D, detect whether a number of frames in the delay queue is greater than a preset value, and perform step E when the number of frames in the delay queue is greater than the preset value;
a copy module, configured to perform step E, copy the data of the target frame into a decoder frame buffer space;
a display module, configured to perform step F, and output the target frame for display;
and a repeating module for repeating the steps A to F until the video data is decoded.
9. A storage medium having a computer program stored thereon, the computer program, when being executed by a processor, performing the steps of the method according to any one of claims 1 to 7.
10. A terminal comprising a memory and a processor, the memory having stored thereon a computer program operable on the processor, wherein the processor, when executing the computer program, performs the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011373449.4A CN112468875B (en) | 2020-11-30 | 2020-11-30 | Display output control method and device of video decoding frame, storage medium and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011373449.4A CN112468875B (en) | 2020-11-30 | 2020-11-30 | Display output control method and device of video decoding frame, storage medium and terminal |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112468875A true CN112468875A (en) | 2021-03-09 |
| CN112468875B CN112468875B (en) | 2022-03-29 |
Family
ID=74806720
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011373449.4A Active CN112468875B (en) | 2020-11-30 | 2020-11-30 | Display output control method and device of video decoding frame, storage medium and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112468875B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115348454A (en) * | 2022-10-17 | 2022-11-15 | 此芯科技(上海)有限公司 | Video transmission processing method and terminal equipment |
| CN115802095A (en) * | 2023-01-06 | 2023-03-14 | 北京象帝先计算技术有限公司 | Video streaming device, system, equipment and video streaming method |
| WO2024007550A1 (en) * | 2022-07-05 | 2024-01-11 | 华为技术有限公司 | Encoding method, decoding method, apparatus, and system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080002773A1 (en) * | 2006-06-26 | 2008-01-03 | Texas Instruments Incorporated | Video decoded picture buffer |
| US20110008022A1 (en) * | 2009-07-13 | 2011-01-13 | Lee Alex Y | System and Methods for Recording a Compressed Video and Audio Stream |
| US20110096833A1 (en) * | 2009-10-22 | 2011-04-28 | Sony Computer Entertainment Inc. | Software video decoder display buffer underflow prediction and recovery |
| US20130003864A1 (en) * | 2011-06-30 | 2013-01-03 | Microsoft Corporation | Reducing latency in video encoding and decoding |
| CN105847823A (en) * | 2016-04-05 | 2016-08-10 | 北京佳讯飞鸿电气股份有限公司 | Method for reducing use of memory bandwidth during video decoding |
| CN107517388A (en) * | 2011-11-25 | 2017-12-26 | 三星电子株式会社 | Method for encoding images and device and picture decoding method and device |
| US9984653B1 (en) * | 2015-02-11 | 2018-05-29 | Synaptics Incorporated | Method and device for reducing video latency |
| CN110832875A (en) * | 2018-07-23 | 2020-02-21 | 深圳市大疆创新科技有限公司 | Video processing method, terminal device and machine-readable storage medium |
-
2020
- 2020-11-30 CN CN202011373449.4A patent/CN112468875B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080002773A1 (en) * | 2006-06-26 | 2008-01-03 | Texas Instruments Incorporated | Video decoded picture buffer |
| US20110008022A1 (en) * | 2009-07-13 | 2011-01-13 | Lee Alex Y | System and Methods for Recording a Compressed Video and Audio Stream |
| US20110096833A1 (en) * | 2009-10-22 | 2011-04-28 | Sony Computer Entertainment Inc. | Software video decoder display buffer underflow prediction and recovery |
| US20130003864A1 (en) * | 2011-06-30 | 2013-01-03 | Microsoft Corporation | Reducing latency in video encoding and decoding |
| CN107517388A (en) * | 2011-11-25 | 2017-12-26 | 三星电子株式会社 | Method for encoding images and device and picture decoding method and device |
| US9984653B1 (en) * | 2015-02-11 | 2018-05-29 | Synaptics Incorporated | Method and device for reducing video latency |
| CN105847823A (en) * | 2016-04-05 | 2016-08-10 | 北京佳讯飞鸿电气股份有限公司 | Method for reducing use of memory bandwidth during video decoding |
| CN110832875A (en) * | 2018-07-23 | 2020-02-21 | 深圳市大疆创新科技有限公司 | Video processing method, terminal device and machine-readable storage medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024007550A1 (en) * | 2022-07-05 | 2024-01-11 | 华为技术有限公司 | Encoding method, decoding method, apparatus, and system |
| CN115348454A (en) * | 2022-10-17 | 2022-11-15 | 此芯科技(上海)有限公司 | Video transmission processing method and terminal equipment |
| CN115348454B (en) * | 2022-10-17 | 2022-12-20 | 此芯科技(上海)有限公司 | Video transmission processing method and terminal equipment |
| CN115802095A (en) * | 2023-01-06 | 2023-03-14 | 北京象帝先计算技术有限公司 | Video streaming device, system, equipment and video streaming method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112468875B (en) | 2022-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112468875B (en) | Display output control method and device of video decoding frame, storage medium and terminal | |
| US10659847B2 (en) | Frame dropping method for video frame and video sending apparatus | |
| CN109379624B (en) | Video processing method and device, electronic equipment and storage medium | |
| US11627369B2 (en) | Video enhancement control method, device, electronic device, and storage medium | |
| US9564108B2 (en) | Video frame processing on a mobile operating system | |
| JP2008504750A5 (en) | ||
| US11153525B2 (en) | Method and device for video enhancement, and electronic device using the same | |
| WO2021057412A1 (en) | Video recording method, device, terminal, and computer-readable storage medium | |
| CN109361950B (en) | Video processing method and device, electronic equipment and storage medium | |
| US20230362388A1 (en) | Systems and methods for deferred post-processes in video encoding | |
| CN111432262A (en) | Page video rendering method and device | |
| US9852677B2 (en) | Dithering for image data to be displayed | |
| US20170287106A1 (en) | Graphics system and method for generating a blended image using content hints | |
| CN109587555B (en) | Video processing method and device, electronic equipment and storage medium | |
| US8994789B2 (en) | Digital video signal, a method for encoding of a digital video signal and a digital video signal encoder | |
| CN111405347B (en) | Picture display method and device, electronic equipment and readable storage medium | |
| CN113064728A (en) | High-load application image display method, terminal and readable storage medium | |
| US10645306B2 (en) | Method for producing media file and electronic device thereof | |
| WO2020107970A1 (en) | Video decoding method and apparatus, electronic device, and storage medium | |
| EP3264284B1 (en) | Data processing method and device | |
| WO2020038071A1 (en) | Video enhancement control method, device, electronic apparatus, and storage medium | |
| CN109859328B (en) | Scene switching method, device, equipment and medium | |
| US9749579B2 (en) | Apparatus and method for frame rate conversion | |
| US20190019316A1 (en) | A mobile device and a method for texture memory optimization thereof | |
| CN109218803B (en) | Video enhancement control method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |