CN107948546A - A kind of low latency video mix device - Google Patents
A kind of low latency video mix device Download PDFInfo
- Publication number
- CN107948546A CN107948546A CN201711096984.8A CN201711096984A CN107948546A CN 107948546 A CN107948546 A CN 107948546A CN 201711096984 A CN201711096984 A CN 201711096984A CN 107948546 A CN107948546 A CN 107948546A
- Authority
- CN
- China
- Prior art keywords
- video
- low latency
- module
- frame
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/04—Synchronising
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/268—Signal distribution or switching
Abstract
The invention discloses a kind of low latency video mix device, include video input interface, several low latency Video processing synchronization unit, video cache module and video mix processing units, video input interface selection all the way input video as main view frequently, other videos are used as from video, and the synchronous sequence signal of the main view frequency received is inputted each low latency Video processing synchronization unit respectively；Each low latency Video processing synchronization unit will be synchronous with main view frequency from video by the implementing reading and writing in video cache module from video according to synchronous sequence signal；Slave video after main view frequency and synchronization is carried out fusion output by video mix processing unit；The video cache module is deposited for the distribution of each low latency Video processing synchronization unit no more than 2 frames.The present invention can make the minimum half for being reduced to conventional method of video delay by the video mix scheme for carrying out 2 frame buffers to video and video rate is predicted.
Description
Technical field
The present invention relates to Video processing and the process circuit of microelectronic, it is related to a kind of by various video mixed processing
Device.
Background technology
In Video processing, often it is related to multi-channel video and carries out mixed processing, such as regard operation interface and camera
Frequency mixing is presented to user, easy to operation and observation.When carrying out multi-channel video mixed processing using programming device, tradition
Method is that input video is adopted integrated image data to be cached and scaled using three frame buffer methods, exported after mixed processing.The party
Method forbids carrying out any frame in three frame buffers while deposit and read operation, the integrality of video can be effectively ensured, very
The mixing of the good nonsynchronous multichannel input video of adaptation, but it is also obvious that the delay of video mix is larger the shortcomings that this method,
The storage resource of consumption is more.The present invention provides a kind of lower, less method for processing video frequency of consumption storage resource that is delayed, especially
Suitable multi-channel video can not ensure synchronization, and to the more sensitive application scenario that is delayed.
The content of the invention
Present invention aims at a kind of low latency video mix device is provided, by carrying out 2 frame buffers and video to video
The video mix scheme of rate prediction, can make the minimum half for being reduced to conventional method of video delay.
The goal of the invention of the present invention is achieved through the following technical solutions：
A kind of low latency video mix device, comprising video input interface, several low latency Video processing synchronization units,
Video cache module and video mix processing unit；
Input video is as main view frequency all the way for video input interface selection, and other videos are used as from video, by what is received
The synchronous sequence signal of main view frequency inputs each low latency Video processing synchronization unit respectively；
Each low latency Video processing synchronization unit will pass through in video cache module according to synchronous sequence signal from video
Implementing reading and writing it is synchronous with main view frequency from video；
Slave video after main view frequency and synchronization is carried out fusion output by video mix processing unit；
Video cache module is deposited for the distribution of each low latency Video processing synchronization unit no more than 2 frames.
Preferably, low latency Video processing synchronization unit include video phase tracking prediction module, access conflict prediction and
Caching selecting module, video acquisition module, video data pretreatment module, frame deposit access controller module, video data processing
Module and audio video synchronization output module；
Video phase tracking prediction module is used to detect from video and the speed difference and phase difference of synchronous sequence signal；
Access conflict is predicted and caching selecting module is used for when consistent with the speed of synchronous sequence signal from video, control
Frame deposits the read pointer of access controller and write pointer is deposited in same frame and moved；When the speed ratio synchronous sequence signal from video
When speed is fast, control frame deposits the write-read pin of access controller and read pointer is deposited in two frames and alternately moved, when write pointer is with reading
Pointer differentiates whether write pointer occurs access conflict when same frame is deposited, if access conflict occurs, write pointer is not
It is mobile；When the speed of the speed ratio synchronous sequence signal from video is slow, control frame deposits the write-read pin of access controller and reading refers to
Pin is deposited in two frames and alternately moved, and when write pointer is deposited with read pointer in same frame, differentiates whether read pointer occurs visit
Ask, if access conflict occurs, read pointer does not move；
Video acquisition module deposits video write-in frame according to the position of write pointer on controller of frame storage；
Audio video synchronization output module is using phase difference as the position of delay read pointer from controller of frame storage by video from frame
Deposit reading.
Brief description of the drawings
Fig. 1 is the structure diagram of low latency video mix device；
Fig. 2 is the structure diagram of low latency Video processing synchronization unit；
Phase difference schematic diagrames of the Fig. 3 between video transmission rate and input video and main synchronization video；
Fig. 4 is the process flow schematic diagram that selecting module was predicted and cached in access conflict.
Embodiment
The present invention is made with reference to the accompanying drawings and examples further detailed.
The present embodiment is illustrated by taking the low latency video mix device being made of fpga chip and memory as an example.
Video input interface, several low latency Video processing synchronization units and video mix processing unit are set in fpga chip, will
Memory is as video cache module.
Video input interface from the multi-channel video of input select all the way the most harsh video of delay requirement as main view frequently,
Remaining a few road video is used as from video.The speed of main view frequency is sent to low latency as synchronous sequence signal and regarded by video input interface
Frequency processing synchronization unit, low latency Video processing synchronization unit will pass through in video cache mould according to synchronous sequence signal from video
The implementing reading and writing of block exports after the mixing of all videos is completed in video mix processing unit after synchronous with main view frequency from video.
Data write-in and reading are all lists the characteristics of sequentially progress when low latency Video processing synchronization unit is handled for video cache
Member carries out buffer management control by video phase tracking prediction, and when predicting, same frame buffer writes and reading data will not
When clashing, handled with regard to controlling directly to read from the frame buffer being currently written into, the traditional three frame buffer sides of this method contrast
Delay low about field duration of method, Video processing and timing synchronization, maximum, which is always prolonged, can be reduced to traditional three frame buffer methods
About half.
As shown in Fig. 2, there is low latency Video processing synchronization unit video phase tracking prediction module, access conflict to predict
And caching selecting module, video acquisition module, video data pretreatment module, frame deposit access controller, video data processing mould
The part such as block, audio video synchronization output module forms.
Video phase tracking prediction module is used to persistently detect speed difference and the phase from vision signal and synchronous video signal
Potential difference.The input signal of video phase tracking prediction module has from vision signal (din [23:0]、hs_in、vs_in、pclk_
In, de_in), synchronous sequence signal (sync_hs, sync_vs, sync_de, sync_pclk), video phase predicting tracing portion
Part clock (mclk).Phase difference between video transmission rate and input video and main synchronization video is as shown in Figure 3.
Wherein Vnum_i is the cycle of vs_in, and Vnum_sync is the cycle of sync_vs, unit be video phase prediction with
Track component clocks (mclk), the more big then speed of Vnum_i and Vnum_o values is slower, and it is faster to be worth smaller then speed.D_io_num is to regard
The time difference of field duration between frequency, D_io_num represents that main view frequency is fast to be positive, slow from video；D_io_num is negative indication main view frequency
Slowly, it is fast from video.D_io_num absolute values show more greatly bigger from the speed difference of video and main view frequency.Dnum then represents that input regards
Phase difference between frequency and main synchronization video, Dnum represent that input video phase lags behind main synchronization video to be positive, and Dnum is negative
Input video phase-lead is represented in main synchronization video, absolute value is bigger, and phase difference is bigger.
Access conflict is predicted and caching selecting module persistence forecasting is acquired video write-in behaviour on same frame buffer zone
Make no to conflict with output video read operation, being selected according to prediction result when completing frame collection or a frame is exported can handle
Total delays time to control is to minimum and will not produce the frame buffer zone of conflict and is operated.Access conflict is according to rate video signal, phase
Position, processing delay carry out comprehensive descision.
Access conflict is predicted and caching selection process flow is as shown in Figure 4：
When identical with main video transmission rate from video, the phase difference between video and output video is fixed value at this time,
Controller of frame storage only uses a frame buffer, realizes that output video exports input video into line delay, and time delay is
Phase difference between video and main view frequency.
When very fast from video transmission rate, frame deposits access controller using double frame buffers, and alternate access frame is respectively adopted and delays
The mode deposited is write from video data, reads output video data.Output video data is wherein read using fixed table tennis behaviour
Make alternately to read two frame buffers.In input video frame section start, whether the frame buffer for judging to use ping-pong operation is defeated
Go out the currently used frame buffer of video, according to from the speed of video and main view frequency, phase difference, currently processed delay, calculate and complete
Phase difference between the clock periodicity C_in of present frame write-in and the clock periodicity C_out for completing present frame output, works as C_in
During less than C_out, represent that prediction result for access conflict occurs, is not made to switch, directly selects what a field duration used
Frame buffering is write, the total delay of the video content inputted at this time reaches maximum, an about frame video transmission times, from next
Frame starts, and delay will be adjusted to lowest latency state.
When main video transmission rate is very fast, frame deposits access controller using double frame buffers, and alternate access frame is respectively adopted and delays
The mode deposited writes inputting video data, reads output video data.Inputting video data is wherein write using fixed table tennis
Two frame buffers are alternately read in operation.When reading output video data, in output video frame section start, judge ping-pong operation i.e.
Whether the frame buffer used is the currently used frame buffer of video input, if the frame buffer that input video is currently used, root
According to the speed of input video and synchronization video, phase difference, currently processed delay, the clock periodicity for completing present frame write-in is calculated
Phase difference between C_in and the clock periodicity C_out for completing present frame output, when C_in is more than C_out, represents prediction knot
Fruit then exports video and former frame buffer is carried out to repeat reading, the video data for ensureing output is same for access conflict occurs
Frame video data.It is about a frame video transmission times that video is exported at this time with input video maximum delay, since next frame, is prolonged
Lowest latency state will be adjusted to late.
Video acquisition is used for the memory access operation that video flowing is converted to order.Video data pretreatment is used for having contracting
The occasion of small application, carries out diminution pretreatment, avoids reducing potential bandwidth shortfall risk during output.Video data processing is used for
There is the occasion of the applications such as enhancing, amplification, read from frame buffer after data carry out video data processing and export synchronization video.
Claims (2)
1. a kind of low latency video mix device, comprising video input interface, several low latency Video processing synchronization units, regards
Frequency cache module and video mix processing unit, it is characterised in that：
Input video is as main view frequency all the way for the video input interface selection, and other videos are used as from video, by what is received
The synchronous sequence signal of main view frequency inputs each low latency Video processing synchronization unit respectively；
Each low latency Video processing synchronization unit will pass through the reading in video cache module according to synchronous sequence signal from video
It is realistic existing synchronous with main view frequency from video；
Slave video after main view frequency and synchronization is carried out fusion output by the video mix processing unit；
The video cache module is deposited for the distribution of each low latency Video processing synchronization unit no more than 2 frames.
2. a kind of low latency video mix device according to claim 1, it is characterised in that the low latency Video processing is same
Walk unit and include video phase tracking prediction module, access conflict prediction and caching selecting module, video acquisition module, video counts
Data preprocess module, frame deposit access controller module, video data processing module and audio video synchronization output module；
Video phase tracking prediction module is used to detect from video and the speed difference and phase difference of synchronous sequence signal；
Access conflict is predicted and caching selecting module is used for when consistent with the speed of synchronous sequence signal from video, and control frame is deposited
The read pointer and write pointer of access controller are deposited in same frame to be moved；When the speed of the speed ratio synchronous sequence signal from video
When fast, control frame deposits the write-read pin of access controller and read pointer is deposited in two frames and alternately moved, when write pointer and read pointer
When same frame is deposited, differentiate whether write pointer occurs access conflict, if access conflict occurs, write pointer does not move
It is dynamic；When the speed of the speed ratio synchronous sequence signal from video is slow, control frame deposits the write-read pin and read pointer of access controller
Deposit in two frames and alternately move, when write pointer is deposited with read pointer in same frame, differentiate whether read pointer occurs access
Conflict, if access conflict occurs, read pointer does not move；
The video acquisition module deposits video write-in frame according to the position of write pointer on controller of frame storage；
The audio video synchronization output module is using phase difference as the position of delay read pointer from controller of frame storage by video from frame
Deposit reading.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711096984.8A CN107948546B (en) | 2017-11-09 | 2017-11-09 | Low-delay video mixing device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711096984.8A CN107948546B (en) | 2017-11-09 | 2017-11-09 | Low-delay video mixing device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107948546A true CN107948546A (en) | 2018-04-20 |
| CN107948546B CN107948546B (en) | 2020-07-31 |
Family
ID=61934672
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711096984.8A Active CN107948546B (en) | 2017-11-09 | 2017-11-09 | Low-delay video mixing device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107948546B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110570441A (en) * | 2019-09-16 | 2019-12-13 | 广州波视信息科技股份有限公司 | Ultra-high definition low-delay video control method and system |
| CN110855909A (en) * | 2019-11-14 | 2020-02-28 | 广州魅视电子科技有限公司 | Video signal seamless low-delay switching method and system |
| CN111277770A (en) * | 2020-01-21 | 2020-06-12 | 中国航空无线电电子研究所 | Generalized airborne video processing system based on FPGA |
| CN112511861A (en) * | 2020-12-03 | 2021-03-16 | 威创集团股份有限公司 | Low-delay video transmission method and system and storage medium thereof |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1607935A2 (en) * | 2004-06-08 | 2005-12-21 | Semiconductor Energy Laboratory Co., Ltd. | Simultaneous reading and writing of video memory, and electroluminescent display device |
| CN101043289A (en) * | 2006-06-16 | 2007-09-26 | 华为技术有限公司 | Method and apparatus for solving read-write collision of memory |
| CN200990651Y (en) * | 2006-12-28 | 2007-12-12 | 康佳集团股份有限公司 | Video frequency division amplifier |
| CN102625110A (en) * | 2012-03-30 | 2012-08-01 | 天津天地伟业物联网技术有限公司 | Caching system and caching method for video data |
| CN102740124A (en) * | 2011-03-31 | 2012-10-17 | 英特赛尔美国股份有限公司 | Video multiplexing |
| CN107277595A (en) * | 2017-07-28 | 2017-10-20 | 京东方科技集团股份有限公司 | A kind of multi-channel video synchronous method and device |
-
2017
- 2017-11-09 CN CN201711096984.8A patent/CN107948546B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1607935A2 (en) * | 2004-06-08 | 2005-12-21 | Semiconductor Energy Laboratory Co., Ltd. | Simultaneous reading and writing of video memory, and electroluminescent display device |
| CN101043289A (en) * | 2006-06-16 | 2007-09-26 | 华为技术有限公司 | Method and apparatus for solving read-write collision of memory |
| CN200990651Y (en) * | 2006-12-28 | 2007-12-12 | 康佳集团股份有限公司 | Video frequency division amplifier |
| CN102740124A (en) * | 2011-03-31 | 2012-10-17 | 英特赛尔美国股份有限公司 | Video multiplexing |
| CN102625110A (en) * | 2012-03-30 | 2012-08-01 | 天津天地伟业物联网技术有限公司 | Caching system and caching method for video data |
| CN107277595A (en) * | 2017-07-28 | 2017-10-20 | 京东方科技集团股份有限公司 | A kind of multi-channel video synchronous method and device |
Non-Patent Citations (2)
| Title |
|---|
| ANNA HAC ET AL: "Buffer Control Scheme in Multimedia Synchronization", 《1997 IEEE INTERNATIONAL PERFORMANCE，COMPUTING AND COMMUNICATIONS CONFERENCE》 * |
| 赵震甲: "使用FIFO完成数据传输与同步（中)", 《中国集成电路》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110570441A (en) * | 2019-09-16 | 2019-12-13 | 广州波视信息科技股份有限公司 | Ultra-high definition low-delay video control method and system |
| CN110855909A (en) * | 2019-11-14 | 2020-02-28 | 广州魅视电子科技有限公司 | Video signal seamless low-delay switching method and system |
| CN110855909B (en) * | 2019-11-14 | 2020-07-03 | 广州魅视电子科技有限公司 | Seamless low-delay switching method for video signals |
| CN111277770A (en) * | 2020-01-21 | 2020-06-12 | 中国航空无线电电子研究所 | Generalized airborne video processing system based on FPGA |
| CN112511861A (en) * | 2020-12-03 | 2021-03-16 | 威创集团股份有限公司 | Low-delay video transmission method and system and storage medium thereof |
| CN112511861B (en) * | 2020-12-03 | 2022-05-03 | 威创集团股份有限公司 | Low-delay video transmission method and system and storage medium thereof |
| WO2022116353A1 (en) * | 2020-12-03 | 2022-06-09 | 威创集团股份有限公司 | Low-latency video transmission method and system therefor, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107948546B (en) | 2020-07-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107948546A (en) | A kind of low latency video mix device | |
| CN105872432B (en) | The apparatus and method of quick self-adapted frame rate conversion | |
| CN104202552B (en) | The method and apparatus that both of which MIPI signals are realized by bridging chip | |
| CN107636631A (en) | Image processing apparatus and image processing method | |
| CN107493448A (en) | Image processing system, method for displaying image and display device | |
| CN111768738B (en) | Circuit design method for reducing refresh rate and saving power consumption of AMOLED display driving chip | |
| CN102402409B (en) | A kind of brush screen method and apparatus | |
| CN103065598A (en) | Control method for preventing liquid crystal display from being blurred | |
| CN104333739A (en) | Echoing device and echoing method for spliced wall system | |
| JPS5892160A (en) | Sampling frequency converter | |
| CN104717444A (en) | Method of automatically converting video of multiple formats to VESA (Video Electronics Standards Association)-protocol 1920*1440-resolution 75Hz-frame rate video | |
| CN104239232B (en) | Ping-Pong cache operation structure based on DPRAM (Dual Port Random Access Memory) in FPGA (Field Programmable Gate Array) | |
| CN104702860B (en) | Video image switching system based on FPGA | |
| CN104244100A (en) | Progress bar control method and device | |
| CN104717470B (en) | The caching and the apparatus and method of display spliced based on two-way camera video | |
| CN109873998A (en) | Infrared video based on multi-level guiding filtering enhances system | |
| CN104731586A (en) | Method for designing editable interface of LCD screen of subway train | |
| CN109587421B (en) | HD-SDI/3G-SDI transceiving and real-time picture-in-picture switching output processing method | |
| CN105204799A (en) | Method for increasing display refreshing rate of multi-channel deep memory logic analyzer | |
| CN109874044A (en) | A kind of data pre-storage method, system, equipment and computer-readable medium | |
| CN106383793B (en) | External device access method and system on chip | |
| CN101547364B (en) | Transport stream generator | |
| CN110288939B (en) | LED display screen constant current driving chip based on control serial transmission | |
| CN103685961B (en) | Real-time processing system for achieving video data synchronization using single-chip SRAM | |
| EP3602313B1 (en) | Synchronized processing of data using a system-on-chip |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |