CN110855907B - Low-delay video overlay frame buffer scheduler based on prediction - Google Patents
Low-delay video overlay frame buffer scheduler based on prediction Download PDFInfo
- Publication number
- CN110855907B CN110855907B CN201910996598.7A CN201910996598A CN110855907B CN 110855907 B CN110855907 B CN 110855907B CN 201910996598 A CN201910996598 A CN 201910996598A CN 110855907 B CN110855907 B CN 110855907B
- Authority
- CN
- China
- Prior art keywords
- video
- frame
- unit
- read
- frame buffer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
- H04N5/445—Receiver circuitry for the reception of television signals according to analogue transmission standards for displaying additional information
Abstract
The invention relates to a low-delay video overlay frame buffer scheduler based on prediction, belonging to the technical field of character video image processing in display equipment. The invention constructs a frame buffer scheduler on FPGA, the frame buffer scheduler adopts the technologies of full parallelism, pipeline calculation and the like, and a reusable modularized design method mainly comprises the units of background video buffer, write-in frame buffer scheduling, storage read-write control, background video detection, read-out frame prediction scheduling, foreground video detection, foreground video buffer, video superposition and the like. The read-out frame prediction scheduling unit further shortens the superposition delay while avoiding frame buffer read-write conflicts through a certain prediction algorithm according to the state information of the three units, namely write-in frame buffer scheduling, background video detection, foreground video detection and the like. The buffer scheduler of the superposition frame has the characteristics of preventing the tearing of the superposition frame, online adjusting and configuring the parameters of the prediction algorithm, and the superposition delay being less than 18 milliseconds.
Description
Technical Field
The invention relates to a low-delay video overlay frame buffer scheduler based on prediction, belonging to the technical field of character video image processing in display equipment.
Background
The head-up display superposes the symbols to be displayed with the forward-looking infrared or the video collected by the nacelle and projects the superposed symbols to the field of view right in front of the driver, and when a task with low visibility such as night navigation is executed, the aircraft operator acquires external information and depends on the background video superposed by the head-up display to a great extent. Therefore, the flight safety condition is greatly influenced by the delay characteristic of the superposed background video, and in the traditional superposition implementation scheme, the video superposition delay is generally more than 40 milliseconds, so that the requirement of the head-up display on superposition low delay cannot be met.
Disclosure of Invention
The invention aims to design a low-delay video overlapping frame buffer scheduler based on prediction so as to avoid tearing of video pictures in the overlapping process and greatly reduce video overlapping delay.
The technical scheme of the invention is as follows:
the low-delay video overlay frame buffer scheduler based on prediction is characterized in that: the device comprises a background video caching unit, a writing frame caching scheduling unit, a storage read-write control unit, a background video detection unit, a reading frame prediction scheduling unit, a foreground video detection unit, a foreground video caching unit and a video overlapping unit;
the input end of the background video cache unit receives an input background video signal, and the output end of the background video cache unit is connected with the input end of the write frame cache scheduling unit; the background video buffer unit realizes the first-level buffer of externally input background video data and outputs the buffered continuous video stream signal to the write-in frame buffer scheduling unit;
the output end of the write frame cache scheduling unit is connected with the storage read-write control unit through a storage read-write bus, so that the input background video is written into the multi-frame cache scheduling of the external storage;
the input end of the background video detection unit receives an input background video signal, and line and field synchronizing signals of the background video are detected to obtain line information written into a frame buffer by the background video in real time;
the input end of the foreground video detection unit receives an input foreground video signal, and line and field synchronizing signals of the foreground video are detected to obtain line number information of the foreground video for video superposition in real time;
the input end of the read frame prediction scheduling unit is connected with the state output ends of the write frame cache scheduling unit, the background video detection unit and the foreground video detection unit through video line number signal lines, and receives the write states of the background video and the foreground video and the current write video frame cache number information; calculating a frame cache number which needs to be read currently according to the writing state of the background video and the current written video frame cache number information;
the input end of the read frame prediction scheduling unit is also connected with the storage read-write control unit through a bus, so that read control on the frame cache data which needs to be read currently and is obtained through resolving is realized;
the output end of the read frame prediction scheduling unit is connected with the video superposition unit through a continuous video stream signal, and the read background video frame cache data is output to the video superposition unit;
the input end of the foreground video caching unit receives an input foreground video signal, primary caching of the input foreground video is realized, and a cached continuous video stream signal is output to the video overlapping unit;
the video overlapping unit is used for overlapping and outputting the background video and the foreground video.
In a further preferred aspect, the scheduler for buffering low-latency video overlay frames based on prediction is characterized in that: the logical architecture of the frame buffer scheduler is based on FPGA implementation.
In a further preferred aspect, the scheduler for buffering low-latency video overlay frames based on prediction is characterized in that: the scheduling algorithm used by the write frame buffer scheduling unit is a three-frame buffer rotation write algorithm: three frame buffer areas are virtualized in the external storage and numbered as 1, 2 and 3, and the write frame buffer scheduling unit circularly writes the background video into the external storage according to the sequence of 1-2-3-1.
In a further preferred aspect, the scheduler for buffering low-latency video overlay frames based on prediction is characterized in that: the reading frame prediction scheduling unit calculates a frame buffer number v which needs to be read currently according to the input line position x' of the background video at the current moment, and the buffer number information u of the current writing video frame fed back by the writing frame buffer scheduling unit through the following processes:
setting a read-write conflict threshold value x, and reading a frame number v ═ u if x' > x; if x' is ≦ x, the read frame number is obtained using the following formula
In a further preferred aspect, the scheduler for buffering low-latency video overlay frames based on prediction is characterized in that: the video overlapping unit uses an overlapping algorithm which is an Alpha aliasing algorithm or a foreground covering overlapping algorithm.
Advantageous effects
The invention has the advantages that the frame buffer scheduler is constructed by the FPGA, the frame buffer scheduler adopts the technologies of computing full parallelism, pipeline computing and the like, and a reusable modularized design method mainly comprises the units of background video buffer, writing frame buffer scheduling, storage read-write control, background video detection, read-out frame prediction scheduling, foreground video detection, foreground video buffer, video superposition and the like. The read-out frame prediction scheduling unit further shortens the superposition delay while avoiding frame buffer read-write conflicts through a certain prediction algorithm according to the state information of the three units, namely write-in frame buffer scheduling, background video detection, foreground video detection and the like. The buffer scheduler for the superimposed frame has the characteristics of preventing the superimposed picture from being torn, and the superimposed delay is less than 18 milliseconds.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a diagram of a prediction-based low-latency video overlay frame buffer scheduler architecture according to the present invention;
fig. 2 is a diagram of an embodiment of a prediction-based low-latency video overlay frame buffer scheduler of the present invention.
Detailed Description
The following further describes embodiments of the present invention with reference to the drawings.
As shown in fig. 1, the frame buffer scheduler includes a background video buffer, a write frame buffer scheduling, a storage read-write control, a background video detection, a read frame prediction scheduling, a foreground video detection, a foreground video buffer, and a video overlay unit. The connection relationship and signal flow between the modules are shown in fig. 1.
The architecture of the prediction-based low-latency video overlay frame buffer scheduler system in this embodiment is shown in fig. 2. In the figure, the solid line represents the transmission of continuous video stream signals, the dotted line represents the transmission of video line number information, the thick arrow represents the transmission of the storage read-write data bus, and the arrow direction of each signal connection represents the main signal flow direction. The resolution of the background video used in this embodiment is 1280x1024 refresh rate 60Hz, the resolution of the foreground video is 1280x1024 refresh rate 60Hz, and the resolution of the superimposed output video is 1280x1024 refresh rate 60 Hz.
The background video cache and the foreground video cache unit are realized by using storage resources inside the FPGA, the unit temporarily stores 3 lines of externally input background and foreground videos respectively, and carries out bit width splicing on input video data, and the splicing is specifically realized by splicing 24-bit RGB single-pixel data into 96-bit data of 4 pixels, so that the subsequent storage read-write control unit can be conveniently used.
The background video and foreground video detection unit is realized by using logic codes and is used for acquiring the row and column coordinate values of the input video and sending the related row number information serving as a parameter to the read frame prediction scheduling unit.
And the writing frame buffer scheduling unit sequentially writes the video data into an external storage through the storage read-write control unit according to the detected background video buffer condition and a writing frame buffer scheduling algorithm. The writing frame buffer scheduling algorithm firstly virtualizes three frame buffer spaces of the external storage device, and numbers the three frame buffer spaces as 1, 2 and 3, then starts the writing operation in real time according to the internal state of the background video buffer unit, and the sequence of writing frame buffer is 1-2-3-1, belonging to continuous rolling covering writing.
The read-out frame prediction scheduling unit executes a one-time prediction algorithm at the initial position of the foreground video, and the algorithm deduces the number v of the superimposed frame to be read according to the input line position x' of the background video at the current moment and the current write-in frame number information u fed back by the write-in frame cache scheduling unit, wherein the specific reasoning rule is as follows.
a) Acquiring input line value information x' of a current background video, and acquiring a frame cache number u written by a writing frame cache scheduling unit;
b) setting a read-write conflict threshold value x;
c) predicting whether a read-write conflict occurs to a frame which is being directly read and written by integrating the information, and if the input line value of the background video is greater than a read-write conflict threshold value, namely x' > x, reading a frame number v ═ u; if x' is less than or equal to x, the reading frame number is obtained by the following formula
And then the read frame prediction scheduling unit performs read control by using the storage read-write control unit through the bus according to the frame buffer number which is obtained by calculation and needs to be read currently, and outputs the frame buffer number to the video overlapping unit.
And the video overlapping unit is used for realizing the overlapping and the output of the background and the foreground video. The overlay algorithm used by the video overlay unit may be Alpha aliasing, foreground coverage overlay, and the like, and the overlay algorithm is implemented by using Alpha aliasing in the present embodiment.
The storage read-write control unit is realized by using different read-write control codes according to different external storage physical devices, in the embodiment, the external storage device is DDR3, so the storage read-write control unit realizes the read-write control of a plurality of virtual frame caches by using a DDR3 controller IP.
Through the operation test of the embodiment, the superposition frame buffer scheduler is confirmed to have effective function and excellent performance, the video superposition delay can be shortened to be within 18 milliseconds, and the problems of picture tearing and the like can be effectively avoided.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made in the above embodiments by those of ordinary skill in the art without departing from the principle and spirit of the present invention.
Claims (3)
1. A prediction-based low-latency video overlay frame buffer scheduler, characterized by: the device comprises a background video caching unit, a writing frame caching scheduling unit, a storage read-write control unit, a background video detection unit, a reading frame prediction scheduling unit, a foreground video detection unit, a foreground video caching unit and a video overlapping unit;
the input end of the background video cache unit receives an input background video signal, and the output end of the background video cache unit is connected with the input end of the write frame cache scheduling unit; the background video buffer unit realizes the first-level buffer of externally input background video data and outputs the buffered continuous video stream signal to the write-in frame buffer scheduling unit;
the output end of the write frame cache scheduling unit is connected with the storage read-write control unit through a storage read-write bus, so that the input background video is written into the multi-frame cache scheduling of the external storage; the scheduling algorithm used by the write frame buffer scheduling unit is a three-frame buffer rotation write algorithm: virtualizing three frame buffer areas in external storage, numbering the three frame buffer areas as 1, 2 and 3, and writing the background video into the external storage circularly according to the sequence of 1-2-3-1 by a writing frame buffer scheduling unit;
the input end of the background video detection unit receives an input background video signal, and line and field synchronizing signals of the background video are detected to obtain line information written into a frame buffer by the background video in real time;
the input end of the foreground video detection unit receives an input foreground video signal, and line and field synchronizing signals of the foreground video are detected to obtain line number information of the foreground video for video superposition in real time;
the input end of the read frame prediction scheduling unit is connected with the state output ends of the write frame cache scheduling unit, the background video detection unit and the foreground video detection unit through video line number signal lines, and receives the write states of the background video and the foreground video and the current write video frame cache number information; and resolving the frame buffer number which needs to be read currently according to the writing state of the background video and the current written video frame buffer number information:
the reading frame prediction scheduling unit calculates a frame buffer number v which needs to be read currently according to the input line position x' of the background video at the current moment, and the buffer number information u of the current writing video frame fed back by the writing frame buffer scheduling unit through the following processes:
setting a read-write conflict threshold value x, and reading a frame number v ═ u if x' > x; if x' is ≦ x, the read frame number is obtained using the following formula
The input end of the read frame prediction scheduling unit is also connected with the storage read-write control unit through a bus, so that read control on the frame cache data which needs to be read currently and is obtained through resolving is realized;
the output end of the read frame prediction scheduling unit is connected with the video superposition unit through a continuous video stream signal, and the read background video frame cache data is output to the video superposition unit;
the input end of the foreground video caching unit receives an input foreground video signal, primary caching of the input foreground video is realized, and a cached continuous video stream signal is output to the video overlapping unit;
the video overlapping unit is used for overlapping and outputting the background video and the foreground video.
2. The prediction-based low-latency video overlay frame buffer scheduler of claim 1, wherein: the logical architecture of the frame buffer scheduler is based on FPGA implementation.
3. The prediction-based low-latency video overlay frame buffer scheduler of claim 1, wherein: the video overlapping unit uses an overlapping algorithm which is an Alpha aliasing algorithm or a foreground covering overlapping algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910996598.7A CN110855907B (en) | 2019-10-19 | 2019-10-19 | Low-delay video overlay frame buffer scheduler based on prediction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910996598.7A CN110855907B (en) | 2019-10-19 | 2019-10-19 | Low-delay video overlay frame buffer scheduler based on prediction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110855907A CN110855907A (en) | 2020-02-28 |
| CN110855907B true CN110855907B (en) | 2022-01-04 |
Family
ID=69597681
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910996598.7A Active CN110855907B (en) | 2019-10-19 | 2019-10-19 | Low-delay video overlay frame buffer scheduler based on prediction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110855907B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112235518B (en) * | 2020-10-14 | 2023-02-03 | 天津津航计算技术研究所 | Digital video image fusion and superposition method |
| CN112843677B (en) * | 2021-02-08 | 2023-07-11 | 脸萌有限公司 | System delay determining method, device, equipment and storage medium |
| CN114115720B (en) * | 2021-10-07 | 2024-04-09 | 中国航空工业集团公司洛阳电光设备研究所 | High-frame-rate low-delay graph generating device based on FPGA |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2683155A1 (en) * | 2012-07-07 | 2014-01-08 | PIXarithmic GmbH | System and Method for Processing Video and/or Audio Signals |
| CN104660918B (en) * | 2015-03-11 | 2018-09-21 | 南京航空航天大学 | A kind of real-time mixed display system of Airborne Video System |
| CN109587500B (en) * | 2018-11-26 | 2021-01-01 | 中国航空工业集团公司洛阳电光设备研究所 | FPGA-based dynamically reconfigurable video scaler |
-
2019
- 2019-10-19 CN CN201910996598.7A patent/CN110855907B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110855907A (en) | 2020-02-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110855907B (en) | Low-delay video overlay frame buffer scheduler based on prediction | |
| US10360832B2 (en) | Post-rendering image transformation using parallel image transformation pipelines | |
| EP3491489B1 (en) | Systems and methods for reducing motion-to-photon latency and memory bandwidth in a virtual reality system | |
| EP2893529B1 (en) | Method and device for selective display refresh | |
| US6078339A (en) | Mutual exclusion of drawing engine execution on a graphics device | |
| CN102117594B (en) | Techniques for aligning frame data | |
| US7394465B2 (en) | Displaying an image using memory control unit | |
| US11270492B2 (en) | Graphics processing systems | |
| US11127110B2 (en) | Data processing systems | |
| US11562701B2 (en) | Data processing systems | |
| CN111064906A (en) | Domestic processor and domestic FPGA multi-path 4K high-definition video comprehensive display method | |
| US10890966B2 (en) | Graphics processing systems | |
| US11004427B2 (en) | Method of and data processing system for providing an output surface | |
| JP2004280125A (en) | Video/graphic memory system | |
| CA2855819C (en) | Graphics processing apparatus, display apparatus for an aircraft cockpit, and method for displaying graphical data | |
| US10672367B2 (en) | Providing data to a display in data processing systems | |
| WO2021124729A1 (en) | Display control system, moving object, display control method, display device, display method, and program | |
| CN112422769B (en) | Head-up display for realizing synchronous recording control of characters and videos | |
| US10332489B2 (en) | Data processing system for display underrun recovery | |
| CN114115720A (en) | High frame rate low delay figure generating device based on FPGA | |
| KR20130021988A (en) | Device for overlaying image obtained by camera for vehicle and method thereof | |
| CN110659005B (en) | Operating data processing system and method, display device, and computer readable medium | |
| JP4655473B2 (en) | Vehicle surrounding image conversion device | |
| US11830234B2 (en) | Method and apparatus of processing image | |
| US7999815B1 (en) | Active raster composition and error checking in hardware |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |