US20180018999A1 - Video processing system using ring buffer and racing-mode ring buffer access control scheme - Google Patents
Video processing system using ring buffer and racing-mode ring buffer access control scheme Download PDFInfo
- Publication number
- US20180018999A1 US20180018999A1 US15/646,095 US201715646095A US2018018999A1 US 20180018999 A1 US20180018999 A1 US 20180018999A1 US 201715646095 A US201715646095 A US 201715646095A US 2018018999 A1 US2018018999 A1 US 2018018999A1
- Authority
- US
- United States
- Prior art keywords
- video
- data
- ring buffer
- write
- buffer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
- G11B20/10—Digital recording or reproducing
- G11B20/10527—Audio or video recording; Data buffering arrangements
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
- G09G5/39—Control of the bit-mapped memory
- G09G5/393—Arrangements for updating the contents of the bit-mapped memory
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4341—Demultiplexing of audio and video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
- H04N21/4343—Extraction or processing of packetized elementary streams [PES]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/435—Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network
- H04N21/43632—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network involving a wired protocol, e.g. IEEE 1394
- H04N21/43635—HDMI
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network
- H04N21/43637—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2360/00—Aspects of the architecture of display systems
- G09G2360/12—Frame memory handling
- G09G2360/127—Updating a frame memory using a transfer of data from a source area to a destination area
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
- G11B20/10—Digital recording or reproducing
- G11B20/10527—Audio or video recording; Data buffering arrangements
- G11B2020/1062—Data buffering arrangements, e.g. recording or playback buffers
- G11B2020/10629—Data buffering arrangements, e.g. recording or playback buffers the buffer having a specific structure
- G11B2020/10666—Ring buffers, e.g. buffers wherein an iteratively progressing read or write pointer moves back to the beginning of the buffer when reaching the last storage cell
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
- G11B20/10—Digital recording or reproducing
- G11B20/10527—Audio or video recording; Data buffering arrangements
- G11B2020/1062—Data buffering arrangements, e.g. recording or playback buffers
- G11B2020/10675—Data buffering arrangements, e.g. recording or playback buffers aspects of buffer control
- G11B2020/1074—Data buffering arrangements, e.g. recording or playback buffers aspects of buffer control involving a specific threshold value
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
- G11B20/10—Digital recording or reproducing
- G11B20/10527—Audio or video recording; Data buffering arrangements
- G11B2020/1062—Data buffering arrangements, e.g. recording or playback buffers
- G11B2020/1075—Data buffering arrangements, e.g. recording or playback buffers the usage of the buffer being restricted to a specific kind of data
- G11B2020/10759—Data buffering arrangements, e.g. recording or playback buffers the usage of the buffer being restricted to a specific kind of data content data
- G11B2020/10768—Data buffering arrangements, e.g. recording or playback buffers the usage of the buffer being restricted to a specific kind of data content data by pre-caching the initial portion of songs or other recorded or downloaded data for starting playback instantly
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
- G11B20/10—Digital recording or reproducing
- G11B20/10527—Audio or video recording; Data buffering arrangements
- G11B2020/1062—Data buffering arrangements, e.g. recording or playback buffers
- G11B2020/1075—Data buffering arrangements, e.g. recording or playback buffers the usage of the buffer being restricted to a specific kind of data
- G11B2020/10787—Data buffering arrangements, e.g. recording or playback buffers the usage of the buffer being restricted to a specific kind of data parameters, e.g. for decoding or encoding
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B20/00—Signal processing not specific to the method of recording or reproducing; Circuits therefor
- G11B20/10—Digital recording or reproducing
- G11B20/10527—Audio or video recording; Data buffering arrangements
- G11B2020/1062—Data buffering arrangements, e.g. recording or playback buffers
- G11B2020/1075—Data buffering arrangements, e.g. recording or playback buffers the usage of the buffer being restricted to a specific kind of data
- G11B2020/10796—Data buffering arrangements, e.g. recording or playback buffers the usage of the buffer being restricted to a specific kind of data address data
Definitions
- the disclosed embodiments of the present invention relate to video data processing, and more particularly, to a video processing system using a ring buffer and a racing-mode ring buffer access control scheme.
- One conventional video system design may include a video transmitting system (which may serve as a video recording system) and a video receiving system (which may serve as a video playback system).
- a video transmitting system which may be composed of a plurality of processing stages, including a video encoder, an audio/video multiplexing circuit, a transmitting circuit, etc.
- the video receiving system it may be composed of a plurality of processing stages, including a receiving circuit, an audio/video demultiplexing circuit, a video decoder, a display engine, etc.
- the conventional video system design may fail to meet the requirements of some ultra-low latency applications due to long playback latency at the video receiving system. Hence, there is a need for an innovative low-latency and high-performance video receiving system.
- a video processing system using a ring buffer and a racing-mode ring buffer access control scheme is proposed to solve the above-mentioned problem.
- an exemplary video processing system includes a storage device, an audio/video demultiplexing circuit, and a video decoder.
- the storage device has a bitstream buffer that is a ring buffer.
- the audio/video demultiplexing circuit is arranged to receive an input data, and perform an audio/video demultiplexing operation upon the input data to write data of a video bitstream into the ring buffer.
- the video decoder is arranged to fetch data of the video bitstream from the ring buffer, and perform a video decoding operation upon the fetched data of the video bitstream.
- an exemplary video processing system includes a storage device, a video decoder, and a display engine.
- the storage device includes a display buffer that is a ring buffer.
- the video decoder is arranged to perform a video decoding operation upon data of a video bitstream to write pixel data of a reconstructed video frame into the ring buffer.
- the display engine is arranged to fetch pixel data of the reconstructed video frame from the ring buffer, and drive a display device according to the fetched pixel data of the reconstructed video frame.
- an exemplary video processing system includes a storage device, a receiving circuit, and an audio/video demultiplexing circuit.
- the storage device includes a data buffer that is a ring buffer.
- the receiving circuit is arranged to receive packets and unpack the packets to write payload data of the packets into the ring buffer.
- the audio/video demultiplexing circuit is arranged to fetch an input data from the ring buffer, and perform an audio/video demultiplexing operation upon the fetched input data, wherein the fetched input data comprise payload data of at least one of the packets.
- FIG. 1 is a block diagram illustrating a video receiving system according to an embodiment of the present invention.
- FIG. 2 is a diagram illustrating a first video processing system according to an embodiment of the present invention.
- FIG. 3 is a diagram illustrating a bitstream buffer implemented using a ring buffer according to an embodiment of the present invention.
- FIG. 4 is a flowchart illustrating a method of controlling a write operation of a bitstream buffer according to an embodiment of the present invention.
- FIG. 5 is a flowchart illustrating a method of controlling a read operation of a bitstream buffer according to an embodiment of the present invention.
- FIG. 6 is a diagram illustrating a second video processing system according to an embodiment of the present invention.
- FIG. 7 is a diagram illustrating a display buffer implemented using a ring buffer according to an embodiment of the present invention.
- FIG. 8 is a flowchart illustrating a display control method according to an embodiment of the present invention.
- FIG. 9 is a diagram illustrating a third video processing system according to an embodiment of the present invention.
- FIG. 10 is a diagram illustrating a data buffer implemented using a ring buffer according to an embodiment of the present invention.
- FIG. 11 is a flowchart illustrating a method of controlling a write operation of a data buffer according to an embodiment of the present invention.
- FIG. 12 is a flowchart illustrating a method of controlling a read operation of a data buffer according to an embodiment of the present invention.
- FIG. 13 is a diagram illustrating a frame base address switching circuit according to an embodiment of the present invention.
- FIG. 14 is a diagram illustrating a video frame displayed on a display device according to an embodiment of the present invention.
- FIG. 1 is a block diagram illustrating a video receiving system according to an embodiment of the present invention.
- the video receiving system 120 communicates with a video transmitting system 100 via a communication link 110 .
- the video transmitting system 100 and the video receiving system 120 may be employed by an ultra-low latency application such as a virtual reality (VR) application.
- the video receiving system 120 includes a receiving (RX) circuit 112 , a storage device 124 , an audio/video demultiplexing circuit (denoted by “A/V DEMUX”) 126 , a video decoder (denoted by “VDEC”) 128 , a display engine 130 , and a display control circuit 132 .
- the storage device 124 includes a data buffer 134 , a bitstream buffer 136 , a display buffer 138 , and a reference frame buffer 139 .
- the video transmitting system 100 may serve as a video recording system that is used to encode video frames provided from one or more video sources (not shown) and then transmit encoded video data to the video receiving system 120 via the communication link 110 .
- the video receiving system 120 may serve as a video playback system that is used to receive encode video frame data from the communication link 110 and then decode the encoded video frame data to generate reconstructed video frames to a display device 140 for video playback.
- the display device 140 may be a display screen of a VR headset.
- the communication link 110 may be implemented using a wired communication link or a wireless communication link.
- the receiving circuit 122 receives packets from the communication link 110 , and unpacks the packets to write payload data of the packets into the data buffer 134 .
- the payload data of the packets may include encoded video data, encoded audio data and other user defined data.
- the audio/video demultiplexing circuit 126 fetches an input data from the data buffer 134 , and performs an audio/video demultiplexing operation upon the fetched input data, where the fetched input data may include payload data of at least one of the packets received by the receiving circuit 122 . Due to audio/video demultiplexing, a video bitstream and an audio bitstream are separated and forwarded to the bitstream buffer 136 and the audio data path 133 , respectively.
- the audio/video demultiplexing circuit 126 performs the audio/video demultiplexing operation upon the fetched input data to write data of a video bitstream into the bitstream buffer 136 and supply data of an audio bitstream to the audio data path 133 .
- the audio bitstream is decoded by the audio data path 134 to obtain audio data for audio playback.
- the video decoder 128 fetches data of the video bitstream from the bitstream buffer 136 , and performs a video decoding operation upon the fetched data of the video bitstream to write pixel data of a reconstructed video frame into the display buffer 138 .
- a coding block in a video frame may be encoded using an intra prediction mode or an inter prediction mode.
- a coding block in a current video frame is encoded using the inter prediction mode
- a predicted block used to reconstruct the coding block of the current video frame is found in a reference frame which is a previously reconstructed video frame and stored in the reference frame buffer 139 .
- a reconstructed video frame generated from decoding a previous video frame by the video decoder 128 is further stored in the reference frame buffer 139 to serve as a reference frame that may be used to decode a current video frame.
- the display engine 130 is a driving circuit controlled by the display control circuit 132 .
- the display engine 130 fetches pixel data of the reconstructed video frame from the display buffer 138 , and drives the display device 140 according to the fetched pixel data of the reconstructed video frame.
- the storage device 124 may be implemented using an internal storage device, an external storage device, or a combination of an internal storage device and an external storage device.
- the internal storage device may be a static random access memory (SRAM) or may be flip-flops; and the external storage device may be a dynamic random access memory (DRAM) or may be a flash memory.
- SRAM static random access memory
- DRAM dynamic random access memory
- one buffer is coupled between different processing stages. As shown in FIG. 1 , the data buffer 134 is coupled between the preceding receiving circuit 122 and the following audio/video demultiplexing circuit 126 , the bitstream buffer 136 is coupled between the preceding audio/video demultiplexing circuit 126 and the following video decoder 128 , and the display buffer 138 is coupled between the preceding video decoder 128 and the following display engine 130 . If one buffer coupled between different processing stages with different data processing rates is implemented using a small-sized buffer, the small-sized buffer may suffer from underflow or overflow during processing of one video frame. This may lead to extra access latency.
- the large-sized buffer may avoid underflow and overflow during processing of one video frame at the expense of the buffer cost.
- at least one of data buffer 134 , bitstream buffer 136 , and display buffer 138 may be implemented using a ring buffer. Due to inherent characteristics of a ring buffer, a storage space of a small-sized ring buffer can be reused, such that the small-sized ring buffer may act as a large-sized buffer.
- a racing-mode ring buffer access control scheme may be employed to prevent a following processing stage from retrieving wrong data from the ring buffer and/or prevent a preceding processing stage from overwriting buffered data that is not processed by the following processing stage yet.
- the present invention proposes using a ring buffer between different processing stages in a video receiving system for achieving a balance between the access latency and the buffer cost.
- FIG. 2 is a diagram illustrating a first video processing system according to an embodiment of the present invention.
- the video processing system 200 may be a part of the video receiving system 120 shown in FIG. 1 .
- the video processing system 200 includes the audio/video demultiplexing circuit 126 , the video decoder 128 , and the bitstream buffer 136 .
- the bitstream buffer 136 is implemented using a ring buffer as shown in FIG. 3 .
- a racing-mode ring buffer access control scheme is employed to prevent the video decoder 128 from retrieving wrong data from the ring buffer (i.e., bitstream buffer 136 ), and prevent the audio/video demultiplexing circuit 126 from overwriting buffered data that is not processed by the video decoder 128 yet.
- the bitstream buffer 136 is a ring buffer having a top address v_start and a bottom address v_end.
- the top address v_start and the bottom address v_end may be physical addresses that define a continuous physical storage space.
- the top address v_start and the bottom address v_end may be virtual addresses that define a continuous logical storage space.
- the total bitstream buffer size is equal to (v_start ⁇ v_end).
- the bitstream buffer 136 can be accessed (read/written) in a direction from the top address v_start to the bottom address v_end and then rolling back to the top address v_start from the bottom address v_end.
- the audio/video demultiplexing circuit 126 During audio/video demultiplexing of an input data (which contains encoded video data and encoded audio data) of one video frame, writes data of a video bitstream into the bitstream buffer 136 , such that a write pointer wptr (which is indicative of a current write address of writing data of the video bitstream into the bitstream buffer 136 ) moves downwards. Specifically, the audio/video demultiplexing circuit 126 analyzes the input data (which contains encoded video data and encoded audio data), and writes data of a video bitstream into the bitstream buffer 136 .
- the audio/video demultiplexing circuit 126 updates the write pointer wptr according to the size of video data stored into the bitstream buffer 136 .
- the write pointer wptr reaches the bottom address v_end, the write pointer wptr will roll back to the top address v_start, such that writing data of the video bitstream into the bitstream buffer 136 continues seamlessly.
- the video decoder 128 reads data of the video bitstream from the bitstream buffer 136 , such that a read pointer rptr (which is indicative of a current read address of reading data of the video bitstream from the bitstream buffer 136 ) moves downwards.
- a read pointer rptr which is indicative of a current read address of reading data of the video bitstream from the bitstream buffer 136
- the read pointer wptr will roll back to the top address v_start, such that reading data of the video bitstream from the bitstream buffer 136 continues seamlessly.
- a storage area which buffers the old data segment can be reused to buffer a new data segment generated from the audio/video demultiplexing circuit 126 .
- the storage area can be read by the video decoder 128 again to obtain the new data segment. Due to inherent characteristics of the ring buffer, the write pointer wptr chases the read pointer rptr, and the read pointer rptr also chases the write pointer wptr.
- a racing mode between the read pointer rptr and the write pointer wptr may be employed to control access (read/write) of the bitstream buffer 136 that is a ring buffer.
- the audio/video demultiplexing circuit 126 is configured to include a write controller 202 which updates the write pointer wptr to the video decoder 128
- the video decoder 128 is configured to include a read controller 204 which updates the read pointer rptr to the audio/video demultiplexing circuit 126 .
- FIG. 4 is a flowchart illustrating a method of controlling a write operation of the bitstream buffer 136 according to an embodiment of the present invention. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown in FIG. 4 .
- the method shown in FIG. 4 may be employed by the write controller 202 .
- the write controller 202 sets an initial start point position, wherein a start point is indicative of a start address of a video bitstream stored in the bitstream buffer 136 by the audio/video demultiplexing circuit 126 . Since the bitstream buffer 136 is a ring buffer, the initial start point position is set by an address between the top address v_start and the bottom address v_end.
- a first data segment of the video bitstream is written into a memory word addressed by a write address that is the initial start point position.
- the write controller 202 can continue or resume writing data of the video bitstream into the bitstream buffer 136 (step 408 ).
- the threshold is equal to a step size of incrementing the write pointer wptr.
- this is for illustrative purposes only, and is not meant to be a limitation of the present invention.
- the write controller 202 stops writing data of the video bitstream into the bitstream buffer 136 (step 406 ).
- the racing-mode ring buffer access control scheme prevents the audio/video demultiplexing circuit 126 from overwriting encoded video data that is not processed by the video decoder 128 yet.
- the write controller 202 does not resume writing data of the video bitstream into the bitstream buffer 136 until the video decoder 128 updates the read pointer rptr to a new value.
- the write controller 202 resumes writing data of the video bitstream into the bitstream buffer 136 (steps 404 and 408 ).
- the write controller 202 updates the write pointer wptr to move the write pointer wptr for writing data of the video bitstream into a next write address of the bitstream buffer 136 .
- the write controller 202 updates the write pointer wptr according to the following pseudo code.
- the write controller 202 After data of the video bitstream is written into the bitstream buffer 136 according to the updated write pointer wptr, the write controller 202 checks if there are more data needed to be stored into the bitstream buffer 136 (step 410 ). When there is no data needed to be stored into the bitstream buffer 136 , the write operation is ended. Otherwise, the control flow proceeds with step 404 .
- FIG. 5 is a flowchart illustrating a method of controlling a read operation of the bitstream buffer 136 according to an embodiment of the present invention. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown in FIG. 5 .
- the method shown in FIG. 5 may be employed by the read controller 204 .
- the read controller 204 checks if the start point position is between the top address v_start and the bottom address v_end.
- a start point is indicative of a start address of a video bitstream stored in the bitstream buffer 136 that is a ring buffer.
- the racing-mode ring buffer access control scheme is only active when the start point position is between the top address v_start and the bottom address v_end.
- the control flow proceeds with step 504 when the start point position is between the top address v_start and the bottom address v_end.
- the read controller 204 stops reading data of the video bitstream from the bitstream buffer 136 (step 506 ). In this way, the racing-mode ring buffer access control scheme prevents the video decoder 128 from retrieving wrong encoded video data from the bitstream buffer 136 . The read controller 204 does not resume reading data of the video bitstream from the bitstream buffer 136 until the audio/video demultiplexing circuit 126 updates the write pointer wptr to a new value.
- the read controller 204 resumes reading data of the video bitstream from the bitstream buffer 136 (steps 504 and 508 ).
- the read controller 204 calculates a read length which indicates an amount of data still waiting to be fetched from the bitstream buffer 136 to the video decoder 128 for further processing. For example, the read controller 204 calculates the read length according to the following pseudo code.
- the read controller 204 updates the read pointer rptr to move the read pointer rptr for reading data of the video bitstream from a next read address of the bitstream buffer 136 .
- the read controller 204 updates the read pointer rptr according to the following pseudo code.
- the read controller 204 checks if the frame decode operation is finished (step 512 ). When the frame decode operation is finished, the read operation is ended. Otherwise, the control flow proceeds with step 504 .
- FIG. 6 is a diagram illustrating a second video processing system according to an embodiment of the present invention.
- the video processing system 600 may be a part of the video receiving system 120 shown in FIG. 1 .
- the video processing system 600 includes the video decoder 128 , the display engine 130 , the display control circuit 132 , and the display buffer 138 .
- the display buffer 138 is implemented using a ring buffer as shown in FIG. 7 .
- a racing-mode ring buffer access control scheme is employed to prevent the video decoder 138 from overwriting buffered pixel data that is not used by the display engine 130 for video playback yet.
- the display buffer 138 is a ring buffer having a top address Buf_start and a bottom address Buf_end.
- the top address Buf_start and the bottom address Buf_end may be physical addresses that define a continuous physical storage space.
- the top address Buf_start and the bottom address Buf_end may be virtual addresses that define a continuous logical storage space.
- the total display buffer size is equal to (Buf_start ⁇ Buf_end).
- the total display buffer size may be equal to an amount of pixel data of n (n ⁇ 2) coding block rows, where each coding block contains m (m ⁇ 1) pixel lines.
- a coding block is a basic processing unit of a video coding standard.
- one coding block is one macroblock (MB).
- MB macroblock
- SB super block
- one coding block is one coding tree unit (CTU).
- the display buffer 138 can be accessed (read/written) in a direction from the top address Buf_start to the bottom address Buf_end and then rolling back to the top address Buf_start from the bottom address Buf_end.
- the video decoder 128 writes pixel data of a reconstructed video frame into the display buffer 138 , such that a write address of the pixel data moves downwards.
- the write address reaches the bottom address Buf_end, the write address will roll back to the top address Buf_start, such that writing pixel data of the reconstructed video frame into the display buffer 138 continues seamlessly.
- the display engine 130 reads pixel data of the reconstructed video frame from the display buffer 138 , such that a read address of the pixel data moves downwards.
- the read address reaches the bottom address Buf_end, the read address will roll back to the top address Buf_start, such that reading pixel data of the reconstructed video frame from the display buffer 138 continues seamlessly.
- a storage area which buffers the old data segment can be reused to buffer a new data segment generated from the video decoder 128 .
- the storage area can be read by the display engine 130 again to obtain the new data segment.
- the video playback of a video frame is not started unless some pixel data of the video frame are already available in the display buffer 138 . Due to inherent characteristics of the ring buffer, the write address chases the read address. To achieve smooth video playback, a read operation of the display buffer 138 cannot be paused.
- a racing mode between the read operation and the write operation may be employed to control the write operation of the display buffer 138 that is a ring buffer.
- the video decoder 128 is configured to include a reconstructed line counter 602 which maintains a reconstructed line count value R_Line indicative of the number of pixel lines that are reconstructed by the video decoder 128
- the display engine 130 is configured to include a displayed line counter 604 which maintains a displayed line count value D_Line indicative of the number of pixel lines that are displayed on the display device 140 through the display engine 130 .
- the video decoder 126 is arranged to perform block-based decoding
- the display engine 130 is arranged to perform scan line based displaying.
- the reconstructed line counter 602 increases the reconstructed line count value R_Line by a first increment value (e.g., a value equal to a height of one coding block) each time one coding block row (e.g., MB row, SB row, or CTU row) is completely decoded. For example, assuming that one coding block has N pixels in a column direction, the reconstructed line count value R_Line is updated by R_Line+N when one coding block row is completely decoded.
- the displayed line counter 604 increases the displayed line count value D_Line by a second increment value (e.g., “1”) each time one pixel line is completely displayed. For example, the displayed line count value D_Line is updated by D_Line+1 when one pixel line is completely displayed.
- a second increment value e.g., “1”
- FIG. 8 is a flowchart illustrating a display control method according to an embodiment of the present invention. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown in FIG. 8 .
- the method shown in FIG. 8 may be employed by the display controller 132 .
- the video decoder 128 updates the reconstructed line count value R_Line to the display control circuit 132
- the display engine 130 updates the displayed line count value D_Line to the display control circuit 132 .
- the display control circuit 132 can refer to the reconstructed line count value R_Line to know the decoding status of a current video frame, and can refer to the displayed line count value D_Line to know the display status of the current video frame.
- the display control circuit 132 compares the reconstructed line count value R_Line and the displayed line count value D_Line to determine if video playback of a reconstructed video frame can be started.
- the display control circuit 132 assigns a frame base address ADDR to the display engine (step 802 ), where the frame base address is indicative of a start address of the reconstructed video frame stored in the display buffer 138 by the video decoder 128 .
- the flow proceeds with step 810 , such that the display engine 130 continues reading pixel data of the reconstructed video frame from the display buffer 138 and driving the display device 140 to display the pixel data of the reconstructed video frame.
- the display control circuit 132 asserts the control signal S ctrl for instructing the video decoder 128 to stop writing pixel data of the reconstructed video frame into the display buffer 138 (step 806 ).
- the racing-mode ring buffer access control scheme prevents the video decoder 128 from overwriting pixel data that is not processed by the display engine 130 for video playback yet.
- a read operation of the display buffer 138 cannot be paused for achieving smooth video playback.
- step 808 the display engine 130 continues reading pixel data of the reconstructed video frame from the display buffer 138 and driving the display device 140 to display the pixel data of the reconstructed video frame.
- the display control circuit 132 does not instruct the video decoder 128 to resume writing pixel data of the reconstructed video frame into the display buffer 138 until the display engine 130 (particularly, the displayed line counter 604 ) updates the displayed line count value D_Line to a new value.
- the display control circuit 132 de-asserts the control signal S ctrl for instructing the video decoder 128 to resume writing pixel data of the reconstructed video frame into the display buffer 138 (steps 804 and 810 ).
- step 812 the display control circuit 132 checks if there are more pixel lines needed to be displayed. When no pixel line is needed to be displayed, the display control operation is ended. Otherwise, the control flow proceeds with step 804 .
- FIG. 9 is a diagram illustrating a third video processing system according to an embodiment of the present invention.
- the video processing system 900 may be a part of the video receiving system 120 shown in FIG. 1 .
- the video processing system 900 includes the receiving circuit 122 , the audio/video demultiplexing circuit 126 , and the data buffer 134 .
- the data buffer 134 is implemented using a ring buffer as shown in FIG. 10 .
- a racing-mode ring buffer access control scheme is employed to prevent the audio/video demultiplexing circuit 126 from retrieving wrong data from the ring buffer (i.e., data buffer 134 ), and prevent the receiving circuit 122 from overwriting buffered data that is not processed by the audio/video demultiplexing circuit 126 yet.
- the communication link 110 shown in FIG. 1 may be a wireless communication link.
- the receiving circuit 122 is used to receive the packets from the wireless communication link.
- the receiving circuit 122 may be a Wireless Fidelity (WiFi) receiver.
- WiFi Wireless Fidelity
- the communication link 110 shown in FIG. 1 may be a wired communication link.
- the receiving circuit 122 is used to receive the packets from the wired communication link.
- the receiving circuit 122 may be a Universal Serial Bus (USB) receiver or a High-Definition Multimedia Interface (HDMI) receiver.
- USB Universal Serial Bus
- HDMI High-Definition Multimedia Interface
- the data buffer 134 is a ring buffer having a top address v_start and a bottom address v_end.
- the top address v_start and the bottom address v_end may be physical addresses that define a continuous physical storage space.
- the top address v_start and the bottom address v_end may be virtual addresses that define a continuous logical storage space.
- the total data buffer size is equal to (v_start ⁇ v_end).
- the data buffer 134 can be accessed (read/written) in a direction from the top address v_start to the bottom address v_end and then rolling back to the top address v_start from the bottom address v_end.
- the receiving circuit 122 writes payload data of the packets into the data buffer 134 , such that a write pointer wptr (which is indicative of a current write address of writing payload data of the packets into the data buffer 134 ) moves downwards.
- the receiving circuit 122 receives a packet from the communication link 110 , and unpacks the received packet to write payload data included in the received packet into the data buffer 134 .
- the receiving circuit 122 updates the write pointer wptr according to the number of packets received and the size of payload data included in each packet received.
- the write pointer wptr reaches the bottom address v_end, the write pointer wptr will roll back to the top address v_start, such that writing payload data of the packets into the data buffer 134 continues seamlessly.
- the audio/video demultiplexing circuit 126 reads the input data from the data buffer 134 , such that a read pointer rptr (which is indicative of a current read address of reading the input data from the data buffer 134 ) moves downwards.
- a read pointer rptr which is indicative of a current read address of reading the input data from the data buffer 134
- the read pointer wptr will roll back to the top address v_start, such that reading the input data from the data buffer 134 continues seamlessly.
- a storage area which buffers the old data segment can be reused to buffer a new data segment generated from the receiving circuit 122 .
- the storage area can be read by the audio/video demultiplexing circuit 126 again to obtain the new data segment. Due to inherent characteristics of the ring buffer, the write pointer wptr chases the read pointer rptr, and the read pointer rptr also chases the write pointer wptr.
- a racing mode between the read pointer rptr and the write pointer wptr may be employed to control access (read/write) of the data buffer 134 that is a ring buffer.
- the audio/video demultiplexing circuit 126 is configured to include a write controller 902 which updates the write pointer wptr to the audio/video demultiplexing circuit 126
- the audio/video demultiplexing circuit 126 is configured to include a read controller 904 which updates the read pointer rptr to the receiving circuit 122 .
- FIG. 11 is a flowchart illustrating a method of controlling a write operation of the data buffer 134 according to an embodiment of the present invention. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown in FIG. 11 .
- the method shown in FIG. 11 may be employed by the write controller 902 .
- the write controller 902 sets an initial start point position, wherein a start point is indicative of a start address of payload data stored in the data buffer 134 by the receiving circuit 122 . Since the data buffer 134 is a ring buffer, the initial start point position is set by an address between the top address v_start and the bottom address v_end.
- a first data segment of the payload data is written into a memory word addressed by a write address that is the initial start point position.
- the write controller 902 can continue or resume writing payload data of the packets into the data buffer 134 (step 1108 ).
- the threshold is equal to a step size of incrementing the write pointer wptr.
- this is for illustrative purposes only, and is not meant to be a limitation of the present invention.
- the write controller 902 stops writing payload data of the packets into the data buffer 134 (step 1106 ). In this way, the racing-mode ring buffer access control scheme prevents the receiving circuit 122 from overwriting payload data that is not processed by the audio/video demultiplexing circuit 126 yet.
- the write controller 902 does not resume writing payload data of the packets into the data buffer 134 until the audio/video demultiplexing circuit 126 updates the read pointer rptr to a new value.
- the write controller 902 resumes writing data of the video bitstream into the data buffer 134 (steps 1104 and 1108 ).
- the write controller 902 updates the write pointer wptr to move the write pointer wptr for writing data of the video bitstream into a next write address of the data buffer 134 .
- the write controller 902 updates the write pointer wptr according to the following pseudo code.
- the write controller 902 After payload data is written into the bitstream buffer 136 according to the updated write pointer wptr, the write controller 902 checks if there are more data needed to be stored into the data buffer 134 (step 1110 ). When there is no data needed to be stored into the data buffer 134 , the write operation is ended. Otherwise, the control flow proceeds with step 1104 .
- FIG. 12 is a flowchart illustrating a method of controlling a read operation of the data buffer 134 according to an embodiment of the present invention. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown in FIG. 12 .
- the method shown in FIG. 12 may be employed by the read controller 904 .
- the read controller 904 checks if the start point position is between the top address v_start and the bottom address v_end.
- a start point is indicative of a start address of payload data stored in the data buffer 134 that is a ring buffer.
- the racing-mode ring buffer access control scheme is only active when the start point position is between the top address v_start and the bottom address v_end.
- the control flow proceeds with step 1204 when the start point position is between the top address v_start and the bottom address v_end.
- the read controller 204 can continue or resume reading payload data of the packets from the data buffer 134 (step 1208 ).
- the read controller 904 stops reading payload data of the packets from the data buffer 134 (step 1206 ). In this way, the racing-mode ring buffer access control scheme prevents the audio/video demultiplexing circuit 126 from retrieving wrong payload data from the data buffer 134 . The read controller 904 does not resume reading payload data of the packets from the data buffer 134 until the receiving circuit 122 updates the write pointer wptr to a new value.
- the read controller 904 resumes reading payload data of the packets from the data buffer 134 (steps 1204 and 1208 ).
- the read controller 204 calculates a read length which indicates an amount of data still waiting to be fetched from the data buffer 134 to the audio/video demultiplexing circuit 126 for further processing.
- the read controller 904 calculates the read length according to the following pseudo code.
- the read controller 904 updates the read pointer rptr to move the read pointer rptr for reading payload data of the packets from a next read address of the data buffer 134 .
- the read controller 904 updates the read pointer rptr according to the following pseudo code.
- the read controller 904 checks if the frame audio/video demultiplexing operation is finished (step 1212 ). When the frame audio/video demultiplexing operation is finished, the read operation is ended. Otherwise, the control flow proceeds with step 1204 .
- the video decoder 128 is unable to provide some pixel data of one reconstructed video frame in time.
- the decoding speed becomes unstable because of the bandwidth variation of the communication link (e.g., a wired communication link or a wireless communication link) 110 between the video transmission system. 100 and the video receiving system 120 .
- the reconstructed pixel data of a portion of a reconstructed video frame are not ready at the time the display engine 130 starts driving the display device 140 to display the portion of the reconstructed video frame, a broken picture may be displayed on the display device 140 , thus resulting in a poor video quality.
- the present invention further proposes a frame base address switching scheme.
- FIG. 13 is a diagram illustrating a frame base address switching circuit according to an embodiment of the present invention.
- the frame base address switching circuit 1302 is apart of the display control circuit 132 shown in FIG. 1 and FIG. 6 .
- the display buffer 138 is used to buffer pixel data of a reconstructed frame (which is a current frame being decoded), and the reference frame buffer 139 is used to buffer one or more reference frames (which are previous frames already decoded).
- the frame base address switching circuit 1302 may be implemented using a multiplexer having a first input port used to receive a reconstructed frame base address, a second input port used to receive a reference frame base address, and an output port used to selectively output the reconstructed frame base address or the reference frame base address as the frame base address ADDR used by the display engine 130 .
- the frame base address switching circuit 1302 is controlled by a result of comparing the reconstructed line count value R_Line maintained by the reconstructed line counter 602 and the displayed line count value D_Line maintained by the displayed line counter 604 .
- the frame base address switching circuit 1302 selects the reference frame base address as the frame base address ADDR.
- the frame base address switching circuit 1302 selects the reconstructed frame base address as the frame base address ADDR.
- the reconstructed frame base address is indicative of a start address of a reconstructed video frame (which is a current frame being decoded) stored in the display buffer 138 .
- the reference frame base address is indicative of a start address of a reference video frame (which is a previous frame already decoded) stored in the reference frame buffer 139 .
- the frame base address ADDR is supplied from the display control circuit 132 to the display engine 130 .
- the display engine 130 fetches pixel data of a reconstructed video frame (which is a current frame being decoded) from the display buffer 138 according to the frame base address ADDR, and drives the display device 140 according to the pixel data of the reconstructed video frame (which is a current frame being decoded).
- the display engine 130 fetches pixel data of a reference frame (which is a previous frame already decoded) from the reference frame buffer 139 according to the frame base address ADDR, and drives the display device 140 according to the pixel data of the reference frame (which is a previous frame already decoded).
- FIG. 14 is a diagram illustrating a video frame displayed on the display device 140 according to an embodiment of the present invention.
- the displayed video frame IMG is composed of a partial reconstructed frame and a partial reference frame.
- pixel data of a reconstructed video frame (which is a current frame being decoded) are displayed on a first region of the display device 140 through the display engine 130 using the frame base address ADDR set by the reconstructed frame base address.
- pixel data of a reference frame (which is a previous frame already decoded) are displayed on a second region of the display device 140 through the display engine 130 using the frame base address ADDR set by the reference frame base address.
Abstract
Description
- This application claims the benefit of U.S. provisional application No. 62/361,091, filed on Jul. 12, 2016 and incorporated herein by reference.
- The disclosed embodiments of the present invention relate to video data processing, and more particularly, to a video processing system using a ring buffer and a racing-mode ring buffer access control scheme.
- One conventional video system design may include a video transmitting system (which may serve as a video recording system) and a video receiving system (which may serve as a video playback system). Regarding the video transmitting system, it may be composed of a plurality of processing stages, including a video encoder, an audio/video multiplexing circuit, a transmitting circuit, etc. Regarding the video receiving system, it may be composed of a plurality of processing stages, including a receiving circuit, an audio/video demultiplexing circuit, a video decoder, a display engine, etc. However, the conventional video system design may fail to meet the requirements of some ultra-low latency applications due to long playback latency at the video receiving system. Hence, there is a need for an innovative low-latency and high-performance video receiving system.
- In accordance with exemplary embodiments of the present invention, a video processing system using a ring buffer and a racing-mode ring buffer access control scheme is proposed to solve the above-mentioned problem.
- According to a first aspect of the present invention, an exemplary video processing system is disclosed. The exemplary video processing system includes a storage device, an audio/video demultiplexing circuit, and a video decoder. The storage device has a bitstream buffer that is a ring buffer. The audio/video demultiplexing circuit is arranged to receive an input data, and perform an audio/video demultiplexing operation upon the input data to write data of a video bitstream into the ring buffer. The video decoder is arranged to fetch data of the video bitstream from the ring buffer, and perform a video decoding operation upon the fetched data of the video bitstream.
- According to a second aspect of the present invention, an exemplary video processing system is disclosed. The exemplary video processing system includes a storage device, a video decoder, and a display engine. The storage device includes a display buffer that is a ring buffer. The video decoder is arranged to perform a video decoding operation upon data of a video bitstream to write pixel data of a reconstructed video frame into the ring buffer. The display engine is arranged to fetch pixel data of the reconstructed video frame from the ring buffer, and drive a display device according to the fetched pixel data of the reconstructed video frame.
- According to a third aspect of the present invention, an exemplary video processing system is disclosed. The exemplary video processing system includes a storage device, a receiving circuit, and an audio/video demultiplexing circuit. The storage device includes a data buffer that is a ring buffer. The receiving circuit is arranged to receive packets and unpack the packets to write payload data of the packets into the ring buffer. The audio/video demultiplexing circuit is arranged to fetch an input data from the ring buffer, and perform an audio/video demultiplexing operation upon the fetched input data, wherein the fetched input data comprise payload data of at least one of the packets.
- These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.
-
FIG. 1 is a block diagram illustrating a video receiving system according to an embodiment of the present invention. -
FIG. 2 is a diagram illustrating a first video processing system according to an embodiment of the present invention. -
FIG. 3 is a diagram illustrating a bitstream buffer implemented using a ring buffer according to an embodiment of the present invention. -
FIG. 4 is a flowchart illustrating a method of controlling a write operation of a bitstream buffer according to an embodiment of the present invention. -
FIG. 5 is a flowchart illustrating a method of controlling a read operation of a bitstream buffer according to an embodiment of the present invention. -
FIG. 6 is a diagram illustrating a second video processing system according to an embodiment of the present invention. -
FIG. 7 is a diagram illustrating a display buffer implemented using a ring buffer according to an embodiment of the present invention. -
FIG. 8 is a flowchart illustrating a display control method according to an embodiment of the present invention. -
FIG. 9 is a diagram illustrating a third video processing system according to an embodiment of the present invention. -
FIG. 10 is a diagram illustrating a data buffer implemented using a ring buffer according to an embodiment of the present invention. -
FIG. 11 is a flowchart illustrating a method of controlling a write operation of a data buffer according to an embodiment of the present invention. -
FIG. 12 is a flowchart illustrating a method of controlling a read operation of a data buffer according to an embodiment of the present invention. -
FIG. 13 is a diagram illustrating a frame base address switching circuit according to an embodiment of the present invention. -
FIG. 14 is a diagram illustrating a video frame displayed on a display device according to an embodiment of the present invention. - Certain terms are used throughout the description and following claims to refer to particular components. As one skilled in the art will appreciate, manufacturers may refer to a component by different names. This document does not intend to distinguish between components that differ in name but not function. In the following description and in the claims, the terms “include” and “comprise” are used in an open-ended fashion, and thus should be interpreted to mean “include, but not limited to . . . ”. Also, the term “couple” is intended to mean either an indirect or direct electrical connection. Accordingly, if one device is electrically connected to another device, that connection may be through a direct electrical connection, or through an indirect electrical connection via other devices and connections.
-
FIG. 1 is a block diagram illustrating a video receiving system according to an embodiment of the present invention. Thevideo receiving system 120 communicates with a video transmittingsystem 100 via acommunication link 110. Byway of example, but not imitation, the video transmittingsystem 100 and thevideo receiving system 120 may be employed by an ultra-low latency application such as a virtual reality (VR) application. In this embodiment, thevideo receiving system 120 includes a receiving (RX) circuit 112, astorage device 124, an audio/video demultiplexing circuit (denoted by “A/V DEMUX”) 126, a video decoder (denoted by “VDEC”) 128, adisplay engine 130, and adisplay control circuit 132. Thestorage device 124 includes adata buffer 134, abitstream buffer 136, adisplay buffer 138, and areference frame buffer 139. - The
video transmitting system 100 may serve as a video recording system that is used to encode video frames provided from one or more video sources (not shown) and then transmit encoded video data to thevideo receiving system 120 via thecommunication link 110. Thevideo receiving system 120 may serve as a video playback system that is used to receive encode video frame data from thecommunication link 110 and then decode the encoded video frame data to generate reconstructed video frames to adisplay device 140 for video playback. For example, thedisplay device 140 may be a display screen of a VR headset. In addition, thecommunication link 110 may be implemented using a wired communication link or a wireless communication link. - The
receiving circuit 122 receives packets from thecommunication link 110, and unpacks the packets to write payload data of the packets into thedata buffer 134. The payload data of the packets may include encoded video data, encoded audio data and other user defined data. The audio/video demultiplexing circuit 126 fetches an input data from thedata buffer 134, and performs an audio/video demultiplexing operation upon the fetched input data, where the fetched input data may include payload data of at least one of the packets received by thereceiving circuit 122. Due to audio/video demultiplexing, a video bitstream and an audio bitstream are separated and forwarded to thebitstream buffer 136 and theaudio data path 133, respectively. In other words, the audio/video demultiplexing circuit 126 performs the audio/video demultiplexing operation upon the fetched input data to write data of a video bitstream into thebitstream buffer 136 and supply data of an audio bitstream to theaudio data path 133. The audio bitstream is decoded by theaudio data path 134 to obtain audio data for audio playback. Regarding the video processing and playback, thevideo decoder 128 fetches data of the video bitstream from thebitstream buffer 136, and performs a video decoding operation upon the fetched data of the video bitstream to write pixel data of a reconstructed video frame into thedisplay buffer 138. - A coding block in a video frame may be encoded using an intra prediction mode or an inter prediction mode. When a coding block in a current video frame is encoded using the inter prediction mode, a predicted block used to reconstruct the coding block of the current video frame is found in a reference frame which is a previously reconstructed video frame and stored in the
reference frame buffer 139. Hence, a reconstructed video frame generated from decoding a previous video frame by thevideo decoder 128 is further stored in thereference frame buffer 139 to serve as a reference frame that may be used to decode a current video frame. - The
display engine 130 is a driving circuit controlled by thedisplay control circuit 132. Thedisplay engine 130 fetches pixel data of the reconstructed video frame from thedisplay buffer 138, and drives thedisplay device 140 according to the fetched pixel data of the reconstructed video frame. - In this embodiment, the
storage device 124 may be implemented using an internal storage device, an external storage device, or a combination of an internal storage device and an external storage device. For example, the internal storage device may be a static random access memory (SRAM) or may be flip-flops; and the external storage device may be a dynamic random access memory (DRAM) or may be a flash memory. - Since different processing stages in the
video receiving system 120 may have different data processing rates, one buffer is coupled between different processing stages. As shown inFIG. 1 , thedata buffer 134 is coupled between the preceding receivingcircuit 122 and the following audio/video demultiplexing circuit 126, thebitstream buffer 136 is coupled between the preceding audio/video demultiplexing circuit 126 and the followingvideo decoder 128, and thedisplay buffer 138 is coupled between the precedingvideo decoder 128 and the followingdisplay engine 130. If one buffer coupled between different processing stages with different data processing rates is implemented using a small-sized buffer, the small-sized buffer may suffer from underflow or overflow during processing of one video frame. This may lead to extra access latency. If one buffer coupled between different processing stages with different data processing rates is implemented using a large-sized buffer, the large-sized buffer may avoid underflow and overflow during processing of one video frame at the expense of the buffer cost. In this embodiment, at least one ofdata buffer 134,bitstream buffer 136, anddisplay buffer 138 may be implemented using a ring buffer. Due to inherent characteristics of a ring buffer, a storage space of a small-sized ring buffer can be reused, such that the small-sized ring buffer may act as a large-sized buffer. In addition, a racing-mode ring buffer access control scheme may be employed to prevent a following processing stage from retrieving wrong data from the ring buffer and/or prevent a preceding processing stage from overwriting buffered data that is not processed by the following processing stage yet. To put it simply, the present invention proposes using a ring buffer between different processing stages in a video receiving system for achieving a balance between the access latency and the buffer cost. -
FIG. 2 is a diagram illustrating a first video processing system according to an embodiment of the present invention. Thevideo processing system 200 may be a part of thevideo receiving system 120 shown inFIG. 1 . In this embodiment, thevideo processing system 200 includes the audio/video demultiplexing circuit 126, thevideo decoder 128, and thebitstream buffer 136. In addition, thebitstream buffer 136 is implemented using a ring buffer as shown inFIG. 3 . Further, a racing-mode ring buffer access control scheme is employed to prevent thevideo decoder 128 from retrieving wrong data from the ring buffer (i.e., bitstream buffer 136), and prevent the audio/video demultiplexing circuit 126 from overwriting buffered data that is not processed by thevideo decoder 128 yet. - As shown in
FIG. 3 , thebitstream buffer 136 is a ring buffer having a top address v_start and a bottom address v_end. In one embodiment, the top address v_start and the bottom address v_end may be physical addresses that define a continuous physical storage space. In another embodiment, the top address v_start and the bottom address v_end may be virtual addresses that define a continuous logical storage space. Hence, the total bitstream buffer size is equal to (v_start−v_end). Thebitstream buffer 136 can be accessed (read/written) in a direction from the top address v_start to the bottom address v_end and then rolling back to the top address v_start from the bottom address v_end. - During audio/video demultiplexing of an input data (which contains encoded video data and encoded audio data) of one video frame, the audio/
video demultiplexing circuit 126 writes data of a video bitstream into thebitstream buffer 136, such that a write pointer wptr (which is indicative of a current write address of writing data of the video bitstream into the bitstream buffer 136) moves downwards. Specifically, the audio/video demultiplexing circuit 126 analyzes the input data (which contains encoded video data and encoded audio data), and writes data of a video bitstream into thebitstream buffer 136. Hence, the audio/video demultiplexing circuit 126 updates the write pointer wptr according to the size of video data stored into thebitstream buffer 136. When the write pointer wptr reaches the bottom address v_end, the write pointer wptr will roll back to the top address v_start, such that writing data of the video bitstream into thebitstream buffer 136 continues seamlessly. - During video decoding of a video bitstream of one video frame, the
video decoder 128 reads data of the video bitstream from thebitstream buffer 136, such that a read pointer rptr (which is indicative of a current read address of reading data of the video bitstream from the bitstream buffer 136) moves downwards. When the read pointer rptr reaches the bottom address v_end, the read pointer wptr will roll back to the top address v_start, such that reading data of the video bitstream from thebitstream buffer 136 continues seamlessly. - After an old data segment stored in the
bitstream buffer 136 is read and processed by thevideo decoder 128, a storage area which buffers the old data segment can be reused to buffer a new data segment generated from the audio/video demultiplexing circuit 126. In addition, after the old data segment (which has been read and processed by the video decoder 128) is overwritten by the new data segment generated from the audio/video demultiplexing circuit 126, the storage area can be read by thevideo decoder 128 again to obtain the new data segment. Due to inherent characteristics of the ring buffer, the write pointer wptr chases the read pointer rptr, and the read pointer rptr also chases the write pointer wptr. A racing mode between the read pointer rptr and the write pointer wptr may be employed to control access (read/write) of thebitstream buffer 136 that is a ring buffer. In accordance with the racing-mode ring buffer access control scheme, the audio/video demultiplexing circuit 126 is configured to include awrite controller 202 which updates the write pointer wptr to thevideo decoder 128, and thevideo decoder 128 is configured to include aread controller 204 which updates the read pointer rptr to the audio/video demultiplexing circuit 126. -
FIG. 4 is a flowchart illustrating a method of controlling a write operation of thebitstream buffer 136 according to an embodiment of the present invention. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown inFIG. 4 . The method shown inFIG. 4 may be employed by thewrite controller 202. Atstep 402, thewrite controller 202 sets an initial start point position, wherein a start point is indicative of a start address of a video bitstream stored in thebitstream buffer 136 by the audio/video demultiplexing circuit 126. Since thebitstream buffer 136 is a ring buffer, the initial start point position is set by an address between the top address v_start and the bottom address v_end. In addition, at the beginning of writing data of the video bitstream into thebitstream buffer 136, a first data segment of the video bitstream is written into a memory word addressed by a write address that is the initial start point position. - At
step 404, thewrite controller 202 compares its write pointer wptr with the received read pointer rptr to check if a distance between the write pointer wptr and the read pointer rptr does not reach a threshold (e.g., wptr !=rptr−1). When the distance between the write pointer wptr and the read pointer rptr does not reach the threshold (e.g., wptr !=rptr−1), thewrite controller 202 can continue or resume writing data of the video bitstream into the bitstream buffer 136 (step 408). In this example, the threshold is equal to a step size of incrementing the write pointer wptr. However, this is for illustrative purposes only, and is not meant to be a limitation of the present invention. - When the distance between the write pointer wptr and the read pointer rptr reaches the threshold (e.g., wptr==rptr−1), meaning that the write pointer wptr will catch up the read pointer rptr soon, the
write controller 202 stops writing data of the video bitstream into the bitstream buffer 136 (step 406). In this way, the racing-mode ring buffer access control scheme prevents the audio/video demultiplexing circuit 126 from overwriting encoded video data that is not processed by thevideo decoder 128 yet. Thewrite controller 202 does not resume writing data of the video bitstream into thebitstream buffer 136 until thevideo decoder 128 updates the read pointer rptr to a new value. Hence, when the distance between the write pointer wptr and the read pointer rptr does not reach the threshold (e.g., wptr !=rptr−1) due to the read pointer rptr updated to a new value under a condition that writing data of the video bitstream into thebitstream buffer 136 is paused, thewrite controller 202 resumes writing data of the video bitstream into the bitstream buffer 136 (steps 404 and 408). - At
step 408, thewrite controller 202 updates the write pointer wptr to move the write pointer wptr for writing data of the video bitstream into a next write address of thebitstream buffer 136. For example, thewrite controller 202 updates the write pointer wptr according to the following pseudo code. -
if (wptr==v_end) wptr = v_start else wptr = wptr+1 - After data of the video bitstream is written into the
bitstream buffer 136 according to the updated write pointer wptr, thewrite controller 202 checks if there are more data needed to be stored into the bitstream buffer 136 (step 410). When there is no data needed to be stored into thebitstream buffer 136, the write operation is ended. Otherwise, the control flow proceeds withstep 404. -
FIG. 5 is a flowchart illustrating a method of controlling a read operation of thebitstream buffer 136 according to an embodiment of the present invention. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown inFIG. 5 . The method shown inFIG. 5 may be employed by theread controller 204. Atstep 502, theread controller 204 checks if the start point position is between the top address v_start and the bottom address v_end. As mentioned above, a start point is indicative of a start address of a video bitstream stored in thebitstream buffer 136 that is a ring buffer. The racing-mode ring buffer access control scheme is only active when the start point position is between the top address v_start and the bottom address v_end. Hence, the control flow proceeds withstep 504 when the start point position is between the top address v_start and the bottom address v_end. - At
step 504, theread controller 204 compares its read pointer rptr with the received write pointer wptr to check if the read pointer rptr does not catch up the write pointer wptr (e.g., rptr !=wptr). When the read pointer rptr does not catch up the write pointer wptr (e.g., rptr !=wptr), theread controller 204 can continue or resume reading data of the video bitstream from the bitstream buffer 136 (step 508). - However, when the read point rptr catches up the write pointer wptr (e.g., rptr==wptr), the
read controller 204 stops reading data of the video bitstream from the bitstream buffer 136 (step 506). In this way, the racing-mode ring buffer access control scheme prevents thevideo decoder 128 from retrieving wrong encoded video data from thebitstream buffer 136. Theread controller 204 does not resume reading data of the video bitstream from thebitstream buffer 136 until the audio/video demultiplexing circuit 126 updates the write pointer wptr to a new value. Hence, when the read pointer rptr does not catch up the write pointer wptr (e.g., rptr !=wptr) due to the write pointer wptr updated to a new value under a condition that reading data of the video bitstream from thebitstream buffer 136 is paused, theread controller 204 resumes reading data of the video bitstream from the bitstream buffer 136 (steps 504 and 508). - At
step 508, theread controller 204 calculates a read length which indicates an amount of data still waiting to be fetched from thebitstream buffer 136 to thevideo decoder 128 for further processing. For example, theread controller 204 calculates the read length according to the following pseudo code. -
if (wptr < rptr) length = (v_end-rptr) + (wptr-v_start) else length = wptr-rptr - At
step 510, theread controller 204 updates the read pointer rptr to move the read pointer rptr for reading data of the video bitstream from a next read address of thebitstream buffer 136. For example, theread controller 204 updates the read pointer rptr according to the following pseudo code. -
if (rptr==v_end) rptr = v_start else rptr = rptr+1 - After data of the video bitstream is read from the
bitstream buffer 136 according to the updated read pointer rptr, theread controller 204 checks if the frame decode operation is finished (step 512). When the frame decode operation is finished, the read operation is ended. Otherwise, the control flow proceeds withstep 504. -
FIG. 6 is a diagram illustrating a second video processing system according to an embodiment of the present invention. Thevideo processing system 600 may be a part of thevideo receiving system 120 shown inFIG. 1 . In this embodiment, thevideo processing system 600 includes thevideo decoder 128, thedisplay engine 130, thedisplay control circuit 132, and thedisplay buffer 138. In addition, thedisplay buffer 138 is implemented using a ring buffer as shown inFIG. 7 . A racing-mode ring buffer access control scheme is employed to prevent thevideo decoder 138 from overwriting buffered pixel data that is not used by thedisplay engine 130 for video playback yet. - As shown in
FIG. 7 , thedisplay buffer 138 is a ring buffer having a top address Buf_start and a bottom address Buf_end. In one embodiment, the top address Buf_start and the bottom address Buf_end may be physical addresses that define a continuous physical storage space. In another embodiment, the top address Buf_start and the bottom address Buf_end may be virtual addresses that define a continuous logical storage space. Hence, the total display buffer size is equal to (Buf_start−Buf_end). For example, the total display buffer size may be equal to an amount of pixel data of n (n≧2) coding block rows, where each coding block contains m (m≧1) pixel lines. A coding block is a basic processing unit of a video coding standard. For example, when the video coding standard is H.264, one coding block is one macroblock (MB). For another example, when the video coding standard is VP9, one coding block is one super block (SB). For yet another example, when the video coding standard is HEVC (High Efficiency Video Coding), one coding block is one coding tree unit (CTU). - Since the
display buffer 138 is a ring buffer, thedisplay buffer 138 can be accessed (read/written) in a direction from the top address Buf_start to the bottom address Buf_end and then rolling back to the top address Buf_start from the bottom address Buf_end. - During video decoding of one video frame, the
video decoder 128 writes pixel data of a reconstructed video frame into thedisplay buffer 138, such that a write address of the pixel data moves downwards. When the write address reaches the bottom address Buf_end, the write address will roll back to the top address Buf_start, such that writing pixel data of the reconstructed video frame into thedisplay buffer 138 continues seamlessly. - During video playback of a reconstructed video frame, the
display engine 130 reads pixel data of the reconstructed video frame from thedisplay buffer 138, such that a read address of the pixel data moves downwards. When the read address reaches the bottom address Buf_end, the read address will roll back to the top address Buf_start, such that reading pixel data of the reconstructed video frame from thedisplay buffer 138 continues seamlessly. - After an old data segment stored in the
display buffer 138 is read and processed by thedisplay engine 130, a storage area which buffers the old data segment can be reused to buffer a new data segment generated from thevideo decoder 128. In addition, after the old data segment (which has been read and processed by the display engine 130) is overwritten by the new data segment generated from thevideo decoder 128, the storage area can be read by thedisplay engine 130 again to obtain the new data segment. The video playback of a video frame is not started unless some pixel data of the video frame are already available in thedisplay buffer 138. Due to inherent characteristics of the ring buffer, the write address chases the read address. To achieve smooth video playback, a read operation of thedisplay buffer 138 cannot be paused. Hence, a racing mode between the read operation and the write operation may be employed to control the write operation of thedisplay buffer 138 that is a ring buffer. In accordance with the racing-mode ring buffer access control scheme, thevideo decoder 128 is configured to include areconstructed line counter 602 which maintains a reconstructed line count value R_Line indicative of the number of pixel lines that are reconstructed by thevideo decoder 128, and thedisplay engine 130 is configured to include a displayedline counter 604 which maintains a displayed line count value D_Line indicative of the number of pixel lines that are displayed on thedisplay device 140 through thedisplay engine 130. - In this embodiment, the
video decoder 126 is arranged to perform block-based decoding, and thedisplay engine 130 is arranged to perform scan line based displaying. Hence, the reconstructedline counter 602 increases the reconstructed line count value R_Line by a first increment value (e.g., a value equal to a height of one coding block) each time one coding block row (e.g., MB row, SB row, or CTU row) is completely decoded. For example, assuming that one coding block has N pixels in a column direction, the reconstructed line count value R_Line is updated by R_Line+N when one coding block row is completely decoded. In addition, the displayedline counter 604 increases the displayed line count value D_Line by a second increment value (e.g., “1”) each time one pixel line is completely displayed. For example, the displayed line count value D_Line is updated by D_Line+1 when one pixel line is completely displayed. -
FIG. 8 is a flowchart illustrating a display control method according to an embodiment of the present invention. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown inFIG. 8 . The method shown inFIG. 8 may be employed by thedisplay controller 132. In this example, thevideo decoder 128 updates the reconstructed line count value R_Line to thedisplay control circuit 132, and thedisplay engine 130 updates the displayed line count value D_Line to thedisplay control circuit 132. Hence, thedisplay control circuit 132 can refer to the reconstructed line count value R_Line to know the decoding status of a current video frame, and can refer to the displayed line count value D_Line to know the display status of the current video frame. Initially, thedisplay control circuit 132 compares the reconstructed line count value R_Line and the displayed line count value D_Line to determine if video playback of a reconstructed video frame can be started. When R_Line≧D_Line, thedisplay control circuit 132 assigns a frame base address ADDR to the display engine (step 802), where the frame base address is indicative of a start address of the reconstructed video frame stored in thedisplay buffer 138 by thevideo decoder 128. - At
step 804, thedisplay control circuit 132 compares the reconstructed line count value R_Line and the displayed line count value D_Line to determine if the write operation of thedisplay buffer 138 should be stopped. In this example, thedisplay control circuit 132 checks if a distance between the reconstructed line count value R_Line and the displayed line count value D_Line does not reach a threshold (e.g., R_Line !=D_Line+k), where the threshold k may be set by any positive integer value, depending upon the actual design considerations. When the distance between the reconstructed line count value R_Line and the displayed line count value D_Line does not reach the threshold (e.g., R_Line !=D_Line+k), thedisplay control circuit 132 does not assert a control signal Sctrl for instructing thevideo decoder 128 to stop writing pixel data of the reconstructed video frame into thedisplay buffer 138. Next, the flow proceeds withstep 810, such that thedisplay engine 130 continues reading pixel data of the reconstructed video frame from thedisplay buffer 138 and driving thedisplay device 140 to display the pixel data of the reconstructed video frame. Hence, atstep 810, one pixel line is displayed on thedisplay device 140 through thedisplay engine 130, and the displayedline counter 604 updates the displayed line count value D_Line (e.g., D_Line=D_Line+1) correspondingly. - However, when the distance between the reconstructed line count value R_Line and the displayed line count value D_Line reaches the threshold (e.g., R_Line==D_Line+k), meaning that the video decoding speed is much faster than the video playback speed, the
display control circuit 132 asserts the control signal Sctrl for instructing thevideo decoder 128 to stop writing pixel data of the reconstructed video frame into the display buffer 138 (step 806). In this way, the racing-mode ring buffer access control scheme prevents thevideo decoder 128 from overwriting pixel data that is not processed by thedisplay engine 130 for video playback yet. As mentioned above, a read operation of thedisplay buffer 138 cannot be paused for achieving smooth video playback. Next, the flow proceeds withstep 808, such that thedisplay engine 130 continues reading pixel data of the reconstructed video frame from thedisplay buffer 138 and driving thedisplay device 140 to display the pixel data of the reconstructed video frame. Similarly, atstep 808, the displayedline counter 604 updates the displayed line count value D_Line (e.g., D_Line=D_Line+1) correspondingly. - The
display control circuit 132 does not instruct thevideo decoder 128 to resume writing pixel data of the reconstructed video frame into thedisplay buffer 138 until the display engine 130 (particularly, the displayed line counter 604) updates the displayed line count value D_Line to a new value. Hence, when the distance between the reconstructed line count value R_Line and the displayed line count value D_Line does not reach the threshold (e.g., R_Line !=D_Line+k) due to the displayed line count value D_Line updated to a new value under a condition that writing pixel data of the reconstructed video frame into thedisplay buffer 138 is paused, thedisplay control circuit 132 de-asserts the control signal Sctrl for instructing thevideo decoder 128 to resume writing pixel data of the reconstructed video frame into the display buffer 138 (steps 804 and 810). - At
step 812, thedisplay control circuit 132 checks if there are more pixel lines needed to be displayed. When no pixel line is needed to be displayed, the display control operation is ended. Otherwise, the control flow proceeds withstep 804. -
FIG. 9 is a diagram illustrating a third video processing system according to an embodiment of the present invention. Thevideo processing system 900 may be a part of thevideo receiving system 120 shown inFIG. 1 . In this embodiment, thevideo processing system 900 includes the receivingcircuit 122, the audio/video demultiplexing circuit 126, and thedata buffer 134. In addition, thedata buffer 134 is implemented using a ring buffer as shown inFIG. 10 . Further, a racing-mode ring buffer access control scheme is employed to prevent the audio/video demultiplexing circuit 126 from retrieving wrong data from the ring buffer (i.e., data buffer 134), and prevent the receivingcircuit 122 from overwriting buffered data that is not processed by the audio/video demultiplexing circuit 126 yet. - In one exemplary design, the
communication link 110 shown inFIG. 1 may be a wireless communication link. Hence, the receivingcircuit 122 is used to receive the packets from the wireless communication link. For example, the receivingcircuit 122 may be a Wireless Fidelity (WiFi) receiver. However, this is for illustrative purposes only, and is not meant to be a limitation of the present invention. - In another exemplary design, the
communication link 110 shown inFIG. 1 may be a wired communication link. Hence, the receivingcircuit 122 is used to receive the packets from the wired communication link. For example, the receivingcircuit 122 may be a Universal Serial Bus (USB) receiver or a High-Definition Multimedia Interface (HDMI) receiver. However, this is for illustrative purposes only, and is not meant to be a limitation of the present invention. - As shown in
FIG. 10 , thedata buffer 134 is a ring buffer having a top address v_start and a bottom address v_end. In one embodiment, the top address v_start and the bottom address v_end may be physical addresses that define a continuous physical storage space. In another embodiment, the top address v_start and the bottom address v_end may be virtual addresses that define a continuous logical storage space. Hence, the total data buffer size is equal to (v_start−v_end). Thedata buffer 134 can be accessed (read/written) in a direction from the top address v_start to the bottom address v_end and then rolling back to the top address v_start from the bottom address v_end. - During packet receiving of one video frame, the receiving
circuit 122 writes payload data of the packets into thedata buffer 134, such that a write pointer wptr (which is indicative of a current write address of writing payload data of the packets into the data buffer 134) moves downwards. Specifically, the receivingcircuit 122 receives a packet from thecommunication link 110, and unpacks the received packet to write payload data included in the received packet into thedata buffer 134. Hence, the receivingcircuit 122 updates the write pointer wptr according to the number of packets received and the size of payload data included in each packet received. When the write pointer wptr reaches the bottom address v_end, the write pointer wptr will roll back to the top address v_start, such that writing payload data of the packets into thedata buffer 134 continues seamlessly. - During audio/video demultiplexing of an input data (which includes payload data of at least one of the packets) of one video frame, the audio/
video demultiplexing circuit 126 reads the input data from thedata buffer 134, such that a read pointer rptr (which is indicative of a current read address of reading the input data from the data buffer 134) moves downwards. When the read pointer rptr reaches the bottom address v_end, the read pointer wptr will roll back to the top address v_start, such that reading the input data from thedata buffer 134 continues seamlessly. - After an old data segment stored in the
data buffer 134 is read and processed by the audio/video demultiplexing circuit 126, a storage area which buffers the old data segment can be reused to buffer a new data segment generated from the receivingcircuit 122. In addition, after the old data segment (which has been read and processed by the audio/video demultiplexing circuit 126) is overwritten by the new data segment generated from the receivingcircuit 122, the storage area can be read by the audio/video demultiplexing circuit 126 again to obtain the new data segment. Due to inherent characteristics of the ring buffer, the write pointer wptr chases the read pointer rptr, and the read pointer rptr also chases the write pointer wptr. A racing mode between the read pointer rptr and the write pointer wptr may be employed to control access (read/write) of thedata buffer 134 that is a ring buffer. In accordance with the racing-mode ring buffer access control scheme, the audio/video demultiplexing circuit 126 is configured to include awrite controller 902 which updates the write pointer wptr to the audio/video demultiplexing circuit 126, and the audio/video demultiplexing circuit 126 is configured to include aread controller 904 which updates the read pointer rptr to the receivingcircuit 122. -
FIG. 11 is a flowchart illustrating a method of controlling a write operation of thedata buffer 134 according to an embodiment of the present invention. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown inFIG. 11 . The method shown inFIG. 11 may be employed by thewrite controller 902. Atstep 1102, thewrite controller 902 sets an initial start point position, wherein a start point is indicative of a start address of payload data stored in thedata buffer 134 by the receivingcircuit 122. Since thedata buffer 134 is a ring buffer, the initial start point position is set by an address between the top address v_start and the bottom address v_end. In addition, at the beginning of writing payload data of the packets into thedata buffer 134, a first data segment of the payload data is written into a memory word addressed by a write address that is the initial start point position. - At
step 1104, thewrite controller 902 compares its write pointer wptr with the received read pointer rptr to check if a distance between the write pointer wptr and the read pointer rptr does not reach a threshold (e.g., wptr !=rptr−1). When the distance between the write pointer wptr and the read pointer rptr does not reach the threshold (e.g., wptr !=rptr−1), thewrite controller 902 can continue or resume writing payload data of the packets into the data buffer 134 (step 1108). In this example, the threshold is equal to a step size of incrementing the write pointer wptr. However, this is for illustrative purposes only, and is not meant to be a limitation of the present invention. - However, when the distance between the write pointer wptr and the read pointer rptr reaches the threshold (e.g., wptr==rptr−1), meaning that the write pointer wptr will catch up the read pointer rptr soon, the
write controller 902 stops writing payload data of the packets into the data buffer 134 (step 1106). In this way, the racing-mode ring buffer access control scheme prevents the receivingcircuit 122 from overwriting payload data that is not processed by the audio/video demultiplexing circuit 126 yet. Thewrite controller 902 does not resume writing payload data of the packets into thedata buffer 134 until the audio/video demultiplexing circuit 126 updates the read pointer rptr to a new value. Hence, when the distance between the write pointer wptr and the read pointer rptr does not reach the threshold (e.g., wptr !=rptr−1) due to the read pointer rptr updated to a new value under a condition that writing payload data of the packets into thedata buffer 134 is paused, thewrite controller 902 resumes writing data of the video bitstream into the data buffer 134 (steps 1104 and 1108). - At
step 1108, thewrite controller 902 updates the write pointer wptr to move the write pointer wptr for writing data of the video bitstream into a next write address of thedata buffer 134. For example, thewrite controller 902 updates the write pointer wptr according to the following pseudo code. -
if (wptr==v_end) wptr = v_start else wptr = wptr+1 - After payload data is written into the
bitstream buffer 136 according to the updated write pointer wptr, thewrite controller 902 checks if there are more data needed to be stored into the data buffer 134 (step 1110). When there is no data needed to be stored into thedata buffer 134, the write operation is ended. Otherwise, the control flow proceeds withstep 1104. -
FIG. 12 is a flowchart illustrating a method of controlling a read operation of thedata buffer 134 according to an embodiment of the present invention. Provided that the result is substantially the same, the steps are not required to be executed in the exact order shown inFIG. 12 . The method shown inFIG. 12 may be employed by theread controller 904. Atstep 1202, theread controller 904 checks if the start point position is between the top address v_start and the bottom address v_end. As mentioned above, a start point is indicative of a start address of payload data stored in thedata buffer 134 that is a ring buffer. The racing-mode ring buffer access control scheme is only active when the start point position is between the top address v_start and the bottom address v_end. Hence, the control flow proceeds withstep 1204 when the start point position is between the top address v_start and the bottom address v_end. - At
step 1204, theread controller 904 compares its read pointer rptr with the received write pointer wptr to check if the read pointer rptr does not catch up the write pointer wptr (e.g., rptr !=wptr). When the read pointer rptr does not catch up the write pointer wptr (e.g., rptr !=wptr), theread controller 204 can continue or resume reading payload data of the packets from the data buffer 134 (step 1208). - However, when the read point rptr catches up the write pointer wptr (e.g., rptr==wptr), the
read controller 904 stops reading payload data of the packets from the data buffer 134 (step 1206). In this way, the racing-mode ring buffer access control scheme prevents the audio/video demultiplexing circuit 126 from retrieving wrong payload data from thedata buffer 134. Theread controller 904 does not resume reading payload data of the packets from thedata buffer 134 until the receivingcircuit 122 updates the write pointer wptr to a new value. Hence, when the read pointer rptr does not catch up the write pointer wptr (e.g., rptr !=wptr) due to the write pointer wptr updated to a new value under a condition that reading payload data of the packets from thedata buffer 134 is paused, theread controller 904 resumes reading payload data of the packets from the data buffer 134 (steps 1204 and 1208). - At
step 1208, theread controller 204 calculates a read length which indicates an amount of data still waiting to be fetched from thedata buffer 134 to the audio/video demultiplexing circuit 126 for further processing. For example, theread controller 904 calculates the read length according to the following pseudo code. -
if (wptr < rptr) length = (v_end-rptr) + (wptr-v_start) else length = wptr-rptr - At
step 1210, theread controller 904 updates the read pointer rptr to move the read pointer rptr for reading payload data of the packets from a next read address of thedata buffer 134. For example, theread controller 904 updates the read pointer rptr according to the following pseudo code. -
if (rptr==v_end) rptr = v_start else rptr = rptr+1 - After payload data is read from the
data buffer 134 according to the updated read pointer rptr, theread controller 904 checks if the frame audio/video demultiplexing operation is finished (step 1212). When the frame audio/video demultiplexing operation is finished, the read operation is ended. Otherwise, the control flow proceeds withstep 1204. - To achieve smooth video playback, a read operation of the
display buffer 138 cannot be paused. However, due to certain factors, it is possible that thevideo decoder 128 is unable to provide some pixel data of one reconstructed video frame in time. For example, the decoding speed becomes unstable because of the bandwidth variation of the communication link (e.g., a wired communication link or a wireless communication link) 110 between the video transmission system. 100 and thevideo receiving system 120. When the reconstructed pixel data of a portion of a reconstructed video frame are not ready at the time thedisplay engine 130 starts driving thedisplay device 140 to display the portion of the reconstructed video frame, a broken picture may be displayed on thedisplay device 140, thus resulting in a poor video quality. To prevent thedisplay device 140 from displaying a broken picture, the present invention further proposes a frame base address switching scheme. - Please refer to
FIG. 6 in conjunction withFIG. 13 .FIG. 13 is a diagram illustrating a frame base address switching circuit according to an embodiment of the present invention. The frame baseaddress switching circuit 1302 is apart of thedisplay control circuit 132 shown inFIG. 1 andFIG. 6 . As mentioned above, thedisplay buffer 138 is used to buffer pixel data of a reconstructed frame (which is a current frame being decoded), and thereference frame buffer 139 is used to buffer one or more reference frames (which are previous frames already decoded). In this embodiment, the frame baseaddress switching circuit 1302 may be implemented using a multiplexer having a first input port used to receive a reconstructed frame base address, a second input port used to receive a reference frame base address, and an output port used to selectively output the reconstructed frame base address or the reference frame base address as the frame base address ADDR used by thedisplay engine 130. As shown inFIG. 13 , the frame baseaddress switching circuit 1302 is controlled by a result of comparing the reconstructed line count value R_Line maintained by the reconstructedline counter 602 and the displayed line count value D_Line maintained by the displayedline counter 604. When the reconstructed line count value R_Line is smaller than the displayed line count value D_Line, meaning that thevideo decoder 128 fails to provide the needed pixel line data in time, the frame baseaddress switching circuit 1302 selects the reference frame base address as the frame base address ADDR. When the reconstructed line count value R_Line is not smaller than the displayed line count value D_Line, meaning that thevideo decoder 128 can provide the needed pixel line data in time, the frame baseaddress switching circuit 1302 selects the reconstructed frame base address as the frame base address ADDR. - The reconstructed frame base address is indicative of a start address of a reconstructed video frame (which is a current frame being decoded) stored in the
display buffer 138. The reference frame base address is indicative of a start address of a reference video frame (which is a previous frame already decoded) stored in thereference frame buffer 139. As shown inFIG. 6 , the frame base address ADDR is supplied from thedisplay control circuit 132 to thedisplay engine 130. When the frame base address ADDR is set by the reconstructed frame base address, thedisplay engine 130 fetches pixel data of a reconstructed video frame (which is a current frame being decoded) from thedisplay buffer 138 according to the frame base address ADDR, and drives thedisplay device 140 according to the pixel data of the reconstructed video frame (which is a current frame being decoded). When the frame base address ADDR is set by the reference frame base address, thedisplay engine 130 fetches pixel data of a reference frame (which is a previous frame already decoded) from thereference frame buffer 139 according to the frame base address ADDR, and drives thedisplay device 140 according to the pixel data of the reference frame (which is a previous frame already decoded). - When the decoding speed is stable during video playback of the whole of a current video frame, the
display device 140 will display the current video frame completely. However, when the decoding speed is only stable during video playback of a portion of a current video frame, thedisplay device 140 displays the portion of the current video frame and further displays a portion of a previous video frame.FIG. 14 is a diagram illustrating a video frame displayed on thedisplay device 140 according to an embodiment of the present invention. The displayed video frame IMG is composed of a partial reconstructed frame and a partial reference frame. Before the reconstructed line count value R_Line becomes smaller than the displayed line count value D_Line, pixel data of a reconstructed video frame (which is a current frame being decoded) are displayed on a first region of thedisplay device 140 through thedisplay engine 130 using the frame base address ADDR set by the reconstructed frame base address. After the reconstructed line count value R_Line becomes smaller than the displayed line count value D_Line, pixel data of a reference frame (which is a previous frame already decoded) are displayed on a second region of thedisplay device 140 through thedisplay engine 130 using the frame base address ADDR set by the reference frame base address. With the help of automatically switching from the reconstructed frame base address to the reference frame base address, no broken picture is displayed on thedisplay device 140 when the decoding speed is unstable due to the bandwidth variation between thevideo transmitting system 100 and thevideo receiving system 120. - Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.
Claims (20)
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/646,095 US20180018999A1 (en) | 2016-07-12 | 2017-07-11 | Video processing system using ring buffer and racing-mode ring buffer access control scheme |
TW106123206A TWI666930B (en) | 2016-07-12 | 2017-07-11 | Video processing system using ring buffer and racing-mode ring buffer access control scheme |
CN201810062729.XA CN109246475B (en) | 2016-07-12 | 2018-01-23 | Video processing system |
US16/734,384 US10984832B2 (en) | 2016-07-12 | 2020-01-05 | Video processing system using ring buffer and racing-mode ring buffer access control scheme |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201662361091P | 2016-07-12 | 2016-07-12 | |
US15/646,095 US20180018999A1 (en) | 2016-07-12 | 2017-07-11 | Video processing system using ring buffer and racing-mode ring buffer access control scheme |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/734,384 Division US10984832B2 (en) | 2016-07-12 | 2020-01-05 | Video processing system using ring buffer and racing-mode ring buffer access control scheme |
Publications (1)
Publication Number | Publication Date |
---|---|
US20180018999A1 true US20180018999A1 (en) | 2018-01-18 |
Family
ID=60940727
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/646,095 Abandoned US20180018999A1 (en) | 2016-07-12 | 2017-07-11 | Video processing system using ring buffer and racing-mode ring buffer access control scheme |
US16/734,384 Active US10984832B2 (en) | 2016-07-12 | 2020-01-05 | Video processing system using ring buffer and racing-mode ring buffer access control scheme |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/734,384 Active US10984832B2 (en) | 2016-07-12 | 2020-01-05 | Video processing system using ring buffer and racing-mode ring buffer access control scheme |
Country Status (3)
Country | Link |
---|---|
US (2) | US20180018999A1 (en) |
CN (1) | CN109246475B (en) |
TW (1) | TWI666930B (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220283966A1 (en) * | 2019-08-22 | 2022-09-08 | Ams Ag | Signal processor, processor system and method for transferring data |
WO2023007812A1 (en) * | 2021-07-28 | 2023-02-02 | 日立Astemo株式会社 | In-vehicle processing device |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112286657A (en) * | 2020-10-23 | 2021-01-29 | 海光信息技术股份有限公司 | Electronic device and interrupt processing method |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6009231A (en) * | 1994-09-05 | 1999-12-28 | Sony Corporation | Reproduction of information using a ring buffer with read and write pointers separated from each other by substantially half of the total ring buffer capacity |
US20040141553A1 (en) * | 2003-01-21 | 2004-07-22 | Sandeep Bhatia | Buffer descriptor data structure for communication link between decode and display processes in MPEG decoders |
US20080215644A1 (en) * | 2005-03-21 | 2008-09-04 | Koninklijke Philips Electronics, N.V. | Processing A Data Array With A Meandering Scanning Order Using A Circular Buffer Memory |
US20080303838A1 (en) * | 2007-06-07 | 2008-12-11 | Yamaha Corporation | Image processing apparatus |
US20150155002A1 (en) * | 2011-01-17 | 2015-06-04 | Mediatek Inc. | Method and apparatus for accessing data of multi-tile encoded picture stored in buffering apparatus |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA2156463A1 (en) * | 1994-09-05 | 1996-03-06 | Nobuyuki Aoki | Data reproducing method and data reproducing apparatus |
US7403564B2 (en) * | 2001-11-21 | 2008-07-22 | Vixs Systems, Inc. | System and method for multiple channel video transcoding |
JP2003169327A (en) * | 2001-11-30 | 2003-06-13 | Matsushita Electric Ind Co Ltd | Mpeg-2 video variable-length decoding method on software |
KR100539886B1 (en) * | 2004-09-10 | 2005-12-28 | 삼성전자주식회사 | Digital broadcasting reception apparatus and method therefor |
JP4719167B2 (en) * | 2007-02-19 | 2011-07-06 | 株式会社東芝 | Semiconductor memory information storage device and its write failure countermeasure method |
CN101404741B (en) * | 2008-11-06 | 2011-04-06 | 北京创毅视讯科技有限公司 | Audio playing apparatus and method, and digital television chip |
JP2010160653A (en) * | 2009-01-07 | 2010-07-22 | Renesas Electronics Corp | Ring buffer circuit and control circuit for the same |
CN102110461B (en) * | 2009-12-24 | 2014-07-16 | 晨星软件研发(深圳)有限公司 | Memory controller and control method thereof |
US8493992B2 (en) * | 2010-02-04 | 2013-07-23 | Texas Instruments Incorporated | Interrelated WiFi and USB protocols and other application framework processes, circuits and systems |
US9497466B2 (en) * | 2011-01-17 | 2016-11-15 | Mediatek Inc. | Buffering apparatus for buffering multi-partition video/image bitstream and related method thereof |
CN102625193B (en) * | 2011-01-27 | 2014-07-09 | 郑州威科姆科技股份有限公司 | A method of realizing multimedia file network playing by virtue of auxiliary files |
US8390743B2 (en) * | 2011-03-31 | 2013-03-05 | Intersil Americas Inc. | System and methods for the synchronization and display of video input signals |
DE112013001213T5 (en) * | 2012-02-29 | 2014-12-04 | Mitsubishi Electric Corporation | Data transfer device, data transfer method and data transfer program |
US9753733B2 (en) * | 2012-06-15 | 2017-09-05 | Apple Inc. | Methods, apparatus, and processors for packing multiple iterations of loop in a loop buffer |
CN103034538B (en) * | 2012-12-24 | 2015-06-17 | 珠海拓普智能电气股份有限公司 | Multi-level cache processing method of drive program in embedded type operation system |
US20150206596A1 (en) * | 2014-01-21 | 2015-07-23 | Nvidia Corporation | Managing a ring buffer shared by multiple processing engines |
JP6732534B2 (en) * | 2015-05-29 | 2020-07-29 | キヤノン株式会社 | Data processing device and data processing method |
-
2017
- 2017-07-11 TW TW106123206A patent/TWI666930B/en active
- 2017-07-11 US US15/646,095 patent/US20180018999A1/en not_active Abandoned
-
2018
- 2018-01-23 CN CN201810062729.XA patent/CN109246475B/en active Active
-
2020
- 2020-01-05 US US16/734,384 patent/US10984832B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6009231A (en) * | 1994-09-05 | 1999-12-28 | Sony Corporation | Reproduction of information using a ring buffer with read and write pointers separated from each other by substantially half of the total ring buffer capacity |
US20040141553A1 (en) * | 2003-01-21 | 2004-07-22 | Sandeep Bhatia | Buffer descriptor data structure for communication link between decode and display processes in MPEG decoders |
US20080215644A1 (en) * | 2005-03-21 | 2008-09-04 | Koninklijke Philips Electronics, N.V. | Processing A Data Array With A Meandering Scanning Order Using A Circular Buffer Memory |
US20080303838A1 (en) * | 2007-06-07 | 2008-12-11 | Yamaha Corporation | Image processing apparatus |
US20150155002A1 (en) * | 2011-01-17 | 2015-06-04 | Mediatek Inc. | Method and apparatus for accessing data of multi-tile encoded picture stored in buffering apparatus |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20220283966A1 (en) * | 2019-08-22 | 2022-09-08 | Ams Ag | Signal processor, processor system and method for transferring data |
US11954052B2 (en) * | 2019-08-22 | 2024-04-09 | Ams Ag | Signal processor, processor system and method for transferring data |
WO2023007812A1 (en) * | 2021-07-28 | 2023-02-02 | 日立Astemo株式会社 | In-vehicle processing device |
Also Published As
Publication number | Publication date |
---|---|
TW201813406A (en) | 2018-04-01 |
CN109246475A (en) | 2019-01-18 |
CN109246475B (en) | 2021-09-07 |
US10984832B2 (en) | 2021-04-20 |
TWI666930B (en) | 2019-07-21 |
US20200143836A1 (en) | 2020-05-07 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10984832B2 (en) | Video processing system using ring buffer and racing-mode ring buffer access control scheme | |
US8428126B2 (en) | Image decoding device with parallel processors | |
US5646693A (en) | Memory utilization for video decoding and display with 3:2 pull-down | |
JP3759167B2 (en) | Memory controller that decodes and displays compressed video data | |
KR100504471B1 (en) | Video decoding system | |
US9509992B2 (en) | Video image compression/decompression device | |
US7447266B2 (en) | Decoding device and decoding program for video image data | |
US20180020228A1 (en) | Video processing system with multiple syntax parsing circuits and/or multiple post decoding circuits | |
US10026146B2 (en) | Image processing device including a progress notifier which outputs a progress signal | |
TWI520582B (en) | Apparatus and method for buffering context arrays referenced for performing entropy decoding upon multi-tile encoded picture and entropy decoder | |
KR20070092708A (en) | Local macroblock information buffer | |
US9509940B2 (en) | Image output device, image output method, and recording medium | |
KR19990068991A (en) | Video decoding system | |
US10230948B2 (en) | Video transmitting system with on-the-fly encoding and on-the-fly delivering and associated video receiving system | |
US7822121B2 (en) | Method and/or apparatus for implementing global motion compensation in a video system | |
US9990900B2 (en) | Image processing device and method thereof | |
US6144323A (en) | Method and apparatus for decoding video data | |
CN103124348A (en) | Predictive frame dropping method used in wireless video/audio data transmission | |
US20030123555A1 (en) | Video decoding system and memory interface apparatus | |
US10412390B2 (en) | Video processing system using low-cost video encoding/decoding architecture | |
US20100008642A1 (en) | Video apparatus and method thereof | |
CN107241601B (en) | Image data transmission method, device and terminal | |
US20230421998A1 (en) | Video decoder with inline downscaler | |
KR100504507B1 (en) | Apparatus for video decoding | |
JP2012019462A (en) | Picture decoding device and picture decoding method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MEDIATEK INC., TAIWAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WU, MING-LONG;CHENG, CHIA-YUN;CHANG, YUNG-CHANG;REEL/FRAME:042956/0935 Effective date: 20170627 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |