CN109714548A - A kind of real-time video superposition processing system based on FPGA - Google Patents
A kind of real-time video superposition processing system based on FPGA Download PDFInfo
- Publication number
- CN109714548A CN109714548A CN201811555745.9A CN201811555745A CN109714548A CN 109714548 A CN109714548 A CN 109714548A CN 201811555745 A CN201811555745 A CN 201811555745A CN 109714548 A CN109714548 A CN 109714548A
- Authority
- CN
- China
- Prior art keywords
- video
- signal
- module
- superposition
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
A kind of real-time video superposition processing system based on FPGA proposed by the present invention, including frame buffer module, read-write controller module and superposition calculation module.Frame buffer module carries out caching process to the outer background video signal of input, superposition calculation module is under the control of read-write controller module, outer background video data are read from frame buffer module, the character video frequency signal that external GPU chip provides also is received, calculating is overlapped to the same coordinate pixel of two-way different video；Read-write controller module generates writing pixel coordinate according to the clock signal in outer background video signal and character video frequency signal and reads pixel coordinate.The internal resource of hardware resource utilization FPGA needed for the present invention can be met the requirements, and reduce hardware BOM type and expense；Using the concurrency and configuration flexibility of FPGA, designed, designed audio video synchronization module and superposition calculation module can be realized video overlapping function using the caching of very little, substantially reduce video processing delay, improve real-time.
Description
Technical field
The present invention relates to modern electronic technology fields, and in particular to a kind of real-time video superposition processing system based on FPGA
System.
Background technique
With the development of vehicle-mounted, airborne HUD technology and VR/AR technology, the real-time display demand of digital video is not yet
It is disconnected to improve, but at present in many applications, all there are still the big bring a series of problems of video processing delay.Such as fly
Machine assists night flight using HUD, observes outer what comes into a driver's by HUD and causes pilot that cannot timely respond to external rings if delay is excessive
Border variation, will increase security risk；The for another example driving of automobile HUD auxiliary or reverse image application, excessive, the meeting if video processing is delayed
The judgement of driver is impacted, though make driver can not response external newest road conditions, it is serious or even will cause friendship
Interpreter's event；Also such as VR game, if video processing delay is excessive, phenomena such as it is poor to will cause game experiencing, dizziness.
Traditional video superposition processing is realized using general processors such as CPU or GPU, is had the following problems:
1) video signal collection is cached using multiframe, and the acquisition of 2 to 3 frames is brought to be delayed；
2) video output uses Double buffer, and the output of 1 frame is brought to be delayed；
3) video superposition processing is calculated as unit of whole frame, and whole frame just exports after the completion of calculating, and brings the place of 1 frame
Reason delay.
Therefore, traditional processing method needs the delay of about 4 to 5 frames that could complete entire video superposition processing process.This
Outside, general processor progress video, which is handled toward contact, needs to increase the devices such as dedicated SDRAM, Flash, and hardware spending is larger.
FPGA is a kind of heterogeneous processor chip that can be programmed control, can be realized difference by different codings
Circuit, circuit can be optimized according to actual needs, concurrency and real-time with higher.By FPGA to video
It is handled, is a popular direction of video processing in recent years.Its main feature has:
1) concurrency is strong, and the pipeline processes of video acquisition, video superposition may be implemented；
2) strong real-time, by the frame buffer specially designed, by synchronizing of asynchronous video；
3) flexibly, internal resource is abundant for configuration, increases excessive circuit, reduced cost without in outside.
Video superposition processing is carried out with FPGA, video superposition delay can be reduced, meets the video processing of different field
Real-time demand.
Summary of the invention
To solve the problems, such as existing method for processing video frequency to be delayed, excessive, hardware spending is big, and the present invention proposes that one kind is based on
The real-time video superposition processing system of FPGA.
The technical solution of the present invention is as follows:
A kind of real-time video superposition processing system based on FPGA, it is characterised in that: including frame buffer module, read-write
Controller module and superposition calculation module；
Outer background video signal inputs frame buffer module, and frame buffer module caches the outer background video signal of input
Processing, and under the control of read-write controller module, the video data of caching is exported and gives superposition calculation module, is realized not homologous defeated
The asynchronous video entered synchronizes, and is pre-processed for subsequent superposition calculation；
The superposition calculation module reads outer background video under the control of read-write controller module from frame buffer module
Data, and superposition calculation module also receives the character video frequency signal that external GPU chip provides, superposition calculation module is according to following public affairs
Formula is overlapped calculating: P=(X+Y)-X*Y/255 to the same coordinate pixel of two-way different video, and wherein X is character video frequency picture
Vegetarian refreshments gray value, Y are outer background video pixel gray value, and P is picture photo vegetarian refreshments gray value after superposition；
The read-write controller module receives outer background video signal and the clock signal in character video frequency signal, video
Useful signal, line synchronising signal and field sync signal；Writing pixel is generated according to the clock signal in outer background video signal to sit
Mark, and writing pixel coordinate is exported to frame buffer module；It is generated according to the clock signal in character video frequency signal and reads pixel
Coordinate, and reading pixel coordinate is exported and gives superposition calculation module；The clock signal refers to clock signal, video useful signal,
Line synchronising signal and field sync signal.
Beneficial effect
A kind of real-time video superposition processing system based on FPGA proposed by the present invention is regarded due to being used as only with FPGA
The main processing platform that frequency is synchronous, video is superimposed, the internal resource of required hardware resource utilization FPGA can be met the requirements, and be reduced
Hardware BOM type and expense；Using the concurrency and configuration flexibility of FPGA, designed, designed audio video synchronization module and superposition
Video overlapping function can be realized using the caching of very little in computing module, substantially reduces video processing delay, improves in real time
Property.
Additional aspect and advantage of the invention will be set forth in part in the description, and will partially become from the following description
Obviously, or practice through the invention is recognized.
Detailed description of the invention
Above-mentioned and/or additional aspect of the invention and advantage will become from the description of the embodiment in conjunction with the following figures
Obviously and it is readily appreciated that, in which:
Fig. 1 is signal processing flow schematic diagram of the invention；
Fig. 2 is that frame buffer module of the invention forms figure；
Fig. 3 is that read-write controller module of the invention forms figure；
Fig. 4 is that superposition calculation module of the invention forms figure.
Specific embodiment
The embodiment of the present invention is described below in detail, the embodiment is exemplary, it is intended to it is used to explain the present invention, and
It is not considered as limiting the invention.
Real-time video superposition processing system of one of the present embodiment based on FPGA is as shown in Figure 1, include frame buffer mould
Block, read-write controller module and superposition calculation module.
The main function of the frame buffer module is that (background video is usually video camera or infrared sets external background video
The video of standby shooting) input progress caching process.Outer background video signal inputs frame buffer module, and frame buffer module is to input
Outer background video signal carries out caching process, and under the control of read-write controller module, the video data of caching is exported to folded
Add computing module, realizes that the asynchronous video of not homologous input synchronizes, pre-processed for subsequent superposition calculation.
The main function of the superposition calculation module is to carry out to the same coordinate pixel of two-way different video according to algorithm
Superposition processing.The adder and multiplier that superposition processing is obtained using the DSP resource example inside FPGA, does not increase external expense.
The i.e. described superposition calculation module under the control of read-write controller module, is reading outer background view from frame buffer module
Frequency evidence, and superposition calculation module also receives the character video frequency signal that external GPU chip provides, superposition calculation module is according to following
Formula is overlapped calculating: P=(X+Y)-X*Y/255 to the same coordinate pixel of two-way different video, and wherein X is character video frequency
Pixel gray value, Y are outer background video pixel gray value, and P is picture photo vegetarian refreshments gray value after superposition.
The read-write controller is broadly divided into write-in control logic and read control logic, wherein the master of write-in control logic
Act on be according to the clock signal of outer video input generate writing pixel coordinate, according to outer video pixel clock will one by one as
Prime number evidence is written in frame buffer；The main function of read control logic is to generate reading according to the clock signal of character video frequency input
Pixel coordinate is taken, is read outer video pixel data stored in frame buffer one by one according to the pixel clock of character video frequency folded
Add in module, is overlapped processing.
The read-write controller module receives outer background video signal and the clock signal in character video frequency signal, video
Useful signal, line synchronising signal and field sync signal；Writing pixel is generated according to the clock signal in outer background video signal to sit
Mark, and writing pixel coordinate is exported to frame buffer module；It is generated according to the clock signal in character video frequency signal and reads pixel
Coordinate, and reading pixel coordinate is exported and gives superposition calculation module；The clock signal refers to clock signal, video useful signal,
Line synchronising signal and field sync signal.
The clock signal of outer background video and character video frequency enters in read-write controller, when respective according to two video flowings
Sequence generates the ranks coordinate of write-in and reading respectively, and the pixel clock of background video is used as and writes other than China and foreign countries' background video data
Enter clock, is written in frame buffer and stores point by point according to write-in coordinate；Character video frequency data are using the pixel clock of itself as reading
Take clock, take out outer video data from the appropriate address of frame buffer according to coordinate is read, then by the outer video of same coordinate with
Character video frequency data are sent into superposition calculation module carry out algorithm process pixel-by-pixel according to the beat of character video frequency pixel clock,
Finally according to the pixel clock of character video frequency and the superimposed video of sequential export.
Specific frame buffer module schematic diagram is as shown in Figure 2.Since outer background video and character video frequency are not homologous different
Video is walked, superposition processing has to by synchronization process, and the main function of frame buffer exactly keeps in outer background video, so
Character video frequency can read the outer background video data of respective coordinates pixel afterwards with the timing of itself, for rear class processing.Frame
The main body of caching be using the dual-port SRAM of the BRAM resource example chemical conversion inside FPGA, at the write-in end of dual-port SRAM, according to
The calculated outer background video pixel rank addresses of control logic are written, background video pixel clock is as reference in addition, by picture
Element is written to pixel data in the corresponding address of dual port RAM；End is being read, is being regarded according to the calculated character of read control logic
Frequency pixel rank addresses are read from the corresponding address of dual port RAM outer pixel by pixel using character video frequency pixel clock as reference
Background video data namely character video frequency each clock cycle can get the outer background video picture of same coordinate from frame buffer
Prime number evidence send superposition calculation module to be overlapped processing later.
And read-write controller module diagram is as shown in Figure 3.The major function of read-write controller module is according to outer background
The clock signals such as the row field synchronization (VS/HS) of video and character video frequency, useful signal (DE) calculate separately out just received outer
The coordinate of video and character video frequency pixel data, specific Coordinate calculation method are as follows:
1) line address counter is using field synchronization as starting point, and when each field sync signal arrives, linage-counter is reset, each
When row is synchronized to next, line address counter value adds one；
2) column address counter is synchronous as starting point using row, and when each line synchronising signal arrives, linage-counter is reset, in DE
In the signal effective time, each clock cycle, column address counter adds one；
The rank addresses being calculated according to the method described above are sent in frame buffer for Read-write Catrol.
A kind of superposition calculation module diagram such as Fig. 4 institute of real-time video superposition processing method based on FPGA of the invention
Show.The module major function is to be overlapped the outer background video of same coordinate and character pixel data according to certain algorithm
Processing, later with the superimposed video of the sequential export of character video frequency.Wherein, the algorithm of video superposition is using in FPGA
Adder and multiplier that DSP resource example obtains is realized.
In summary, a kind of real-time video superposition processing system based on FPGA of the present invention, is mainly used for handling video
The character picture and externally input background video picture that module itself generates are overlapped, corresponding step are as follows:
1) the outer video of externally input DVI format is decoded as parallel RGB signals after decoder processes and enters FPGA；
2) in FPGA, frame buffer is carried out by the external video data of BRAM resource inside FPGA；
3) BRAM resource turns to dual-port SRAM form by example in FPGA, and twoport is written by one end in outer video data
SRAM, character picture video reads the outer video data of respective pixel according to video flowing timing from the dual-port SRAM other end, with reality
Existing asynchronous video synchronizes；
4) using the DSP resource construction superposition calculation module inside FPGA, the two-path video after synchronization enters superposition meter
Module is calculated, is handled according to certain superposition algorithm, and export in real time.
The Read-write Catrol mode that the present invention takes due to using frame buffer when depositing, outer background video and character video frequency difference
Maximum delay brought by walking is 1 frame (being calculated as 16.7ms with 60Hz), and all frame buffer, read-write controller, superposition meter
It calculates module etc. to realize by FPGA internal resource, has saved BOM type and cost.
Above-mentioned described embodiment, such as vision signal, the type of refresh rate etc., can be according to the feelings of actual product
Condition is modified, and foregoing description is only illustrated with this.The example implemented above is not to specific restriction of the invention, all and this hair
Bright similar technical solution, all should belong to protection scope of the present invention.
Claims (1)
1. a kind of real-time video superposition processing system based on FPGA, it is characterised in that: including frame buffer module, read-write controller
Module and superposition calculation module；
Outer background video signal inputs frame buffer module, and frame buffer module carries out at caching the outer background video signal of input
Reason, and under the control of read-write controller module, the video data of caching is exported and gives superposition calculation module, realizes not homologous input
Asynchronous video synchronize, pre-processed for subsequent superposition calculation；
The superposition calculation module reads outer background video number under the control of read-write controller module from frame buffer module
According to, and superposition calculation module also receives the character video frequency signal that external GPU chip provides, superposition calculation module is according to the following formula
Calculating: P=(X+Y)-X*Y/255 is overlapped to the same coordinate pixel of two-way different video, wherein X is character video frequency pixel
Point gray value, Y are outer background video pixel gray value, and P is picture photo vegetarian refreshments gray value after superposition；
The read-write controller module receives outer background video signal and the clock signal in character video frequency signal, video are effective
Signal, line synchronising signal and field sync signal；Writing pixel coordinate is generated according to the clock signal in outer background video signal, and
Writing pixel coordinate is exported to frame buffer module；It is generated according to the clock signal in character video frequency signal and reads pixel coordinate,
And reading pixel coordinate is exported and gives superposition calculation module；The clock signal refers to clock signal, video useful signal, and row synchronizes
Signal and field sync signal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811555745.9A CN109714548A (en) | 2018-12-18 | 2018-12-18 | A kind of real-time video superposition processing system based on FPGA |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811555745.9A CN109714548A (en) | 2018-12-18 | 2018-12-18 | A kind of real-time video superposition processing system based on FPGA |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109714548A true CN109714548A (en) | 2019-05-03 |
Family
ID=66255935
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811555745.9A Pending CN109714548A (en) | 2018-12-18 | 2018-12-18 | A kind of real-time video superposition processing system based on FPGA |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109714548A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111683213A (en) * | 2020-06-16 | 2020-09-18 | 中国北方车辆研究所 | Self-adaptive character superposition system and method based on region-of-interest gray level image |
| CN112235518A (en) * | 2020-10-14 | 2021-01-15 | 天津津航计算技术研究所 | Digital video image fusion and superposition method |
| CN113852768A (en) * | 2021-09-24 | 2021-12-28 | 中音讯谷科技有限公司 | Audio and video image intelligent control system based on FPGA technology |
| CN114257704A (en) * | 2021-12-17 | 2022-03-29 | 威创集团股份有限公司 | FPGA-based video superposition method, device, equipment and medium |
| CN115314644A (en) * | 2022-09-16 | 2022-11-08 | 广州市保伦电子有限公司 | Video data processing system based on FPGA |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011127673A1 (en) * | 2010-04-16 | 2011-10-20 | 深圳市创凯电子有限公司 | Method for displaying real-time multiple pictures on full-color led dot matrix and device thereof |
| CN103702040A (en) * | 2013-12-31 | 2014-04-02 | 广州华多网络科技有限公司 | Real-time video graphic decoration superposing processing method and system |
| CN106534722A (en) * | 2016-12-19 | 2017-03-22 | 中国科学院长春光学精密机械与物理研究所 | FPGA based video stream character superimposition processing system and processing method |
| CN108055478A (en) * | 2017-12-18 | 2018-05-18 | 天津津航计算技术研究所 | A kind of multi-channel video superposed transmission method based on FC-AV agreements |
-
2018
- 2018-12-18 CN CN201811555745.9A patent/CN109714548A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011127673A1 (en) * | 2010-04-16 | 2011-10-20 | 深圳市创凯电子有限公司 | Method for displaying real-time multiple pictures on full-color led dot matrix and device thereof |
| CN103702040A (en) * | 2013-12-31 | 2014-04-02 | 广州华多网络科技有限公司 | Real-time video graphic decoration superposing processing method and system |
| CN106534722A (en) * | 2016-12-19 | 2017-03-22 | 中国科学院长春光学精密机械与物理研究所 | FPGA based video stream character superimposition processing system and processing method |
| CN108055478A (en) * | 2017-12-18 | 2018-05-18 | 天津津航计算技术研究所 | A kind of multi-channel video superposed transmission method based on FC-AV agreements |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111683213A (en) * | 2020-06-16 | 2020-09-18 | 中国北方车辆研究所 | Self-adaptive character superposition system and method based on region-of-interest gray level image |
| CN112235518A (en) * | 2020-10-14 | 2021-01-15 | 天津津航计算技术研究所 | Digital video image fusion and superposition method |
| CN113852768A (en) * | 2021-09-24 | 2021-12-28 | 中音讯谷科技有限公司 | Audio and video image intelligent control system based on FPGA technology |
| CN114257704A (en) * | 2021-12-17 | 2022-03-29 | 威创集团股份有限公司 | FPGA-based video superposition method, device, equipment and medium |
| CN114257704B (en) * | 2021-12-17 | 2023-10-10 | 威创集团股份有限公司 | FPGA-based video superposition method, device, equipment and medium |
| CN115314644A (en) * | 2022-09-16 | 2022-11-08 | 广州市保伦电子有限公司 | Video data processing system based on FPGA |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109714548A (en) | A kind of real-time video superposition processing system based on FPGA | |
| US10049642B2 (en) | Sending frames using adjustable vertical blanking intervals | |
| WO2019036105A1 (en) | Post-rendering image transformation using parallel image transformation pipelines | |
| CN105898507B (en) | A kind of vision signal synchronous method and device | |
| US11562701B2 (en) | Data processing systems | |
| CN108241213A (en) | Head-mounted display and its control method | |
| CN103139509B (en) | A kind of OSD based on AXI bus protocol controls display packing and device | |
| CN102117594A (en) | Techniques for aligning frame data | |
| US10861243B1 (en) | Context-sensitive augmented reality | |
| CN101105929A (en) | Data processor | |
| CN109426474B (en) | Double-display-screen synchronization system | |
| CN103927780B (en) | The method and three-dimensional display system that a kind of more video cards render | |
| CN101814269A (en) | Method and device for simultaneously displaying multiple images in real time on full color LED dot matrix | |
| Rensink et al. | Image flicker is as good as saccades in making large scene changes invisible | |
| CN113219668B (en) | Method and device for refreshing screen of head-mounted display device and electronic device | |
| CN108616674A (en) | Two-way video-signal timing sequence generating circuit structure with outer synchronizing function | |
| US10672367B2 (en) | Providing data to a display in data processing systems | |
| CN103327352A (en) | Device and method for achieving double display screen 3D display by adoption of serial processing mode | |
| CN101383913A (en) | Display overlapping control system and control method thereof | |
| CN112486684A (en) | Driving image display method, device and platform, storage medium and embedded equipment | |
| US7425962B2 (en) | Systems and methods for generating a composite video signal from a plurality of independent video signals | |
| CN113315964B (en) | Display method and device of 3D image and electronic equipment | |
| CN115100020A (en) | Virtual, augmented and mixed reality systems and methods | |
| CN116012695A (en) | FPGA implementation method for parallel acquisition and enhancement of multipath infrared images | |
| CN115002304A (en) | Video image resolution self-adaptive conversion device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190503 |
|
| WD01 | Invention patent application deemed withdrawn after publication |