CN104796670A - Cross-platform multiway realtime video processing system architecture based on flow line architecture - Google Patents
Cross-platform multiway realtime video processing system architecture based on flow line architecture Download PDFInfo
- Publication number
- CN104796670A CN104796670A CN201510197275.3A CN201510197275A CN104796670A CN 104796670 A CN104796670 A CN 104796670A CN 201510197275 A CN201510197275 A CN 201510197275A CN 104796670 A CN104796670 A CN 104796670A
- Authority
- CN
- China
- Prior art keywords
- video
- module
- image
- processing system
- cross
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a cross-platform multiway realtime video processing system architecture based on a flow line architecture. The cross-platform multiway realtime video processing system is characterized by comprising a system layer, an abstraction layer and a library layer, a video resource acquiring module automatically reads collected image packet data, and a video image processing module decodes the collected image packet data read by the video resource acquiring module to be corresponding image frames; a control module acquires synchronous video frame sequences from a video synchronizing module and transmits the same to the video image processing module for processing; the control module acquires image video after being processed for an image displaying module to display and starts a video image storing module to store video images at the same time. Compared with the prior art, the cross-platform multiway realtime video processing system has the advantages that the multiway video processing system is abstracted to be a plurality of mutually-independent and mutually-coordinated subsystems, and the subsystems are not interdependent and can realize flexible replacement and combination to meet needs of different application scenes.
Description
Technical field
The present invention relates to video image processing technology, in particular a kind of cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture.
Background technology
Along with the develop rapidly of computer technology, the Capability Requirement of people to electronic equipment process video is also more and more higher.Meanwhile, computer video treatment technology is also applied to increasing field, as education, and commercial advertisement, video display amusement, medical treatment, tourism, human simulation etc.But wish to obtain wider visual angle as people, or during the panoramic video of large scene, the video of single channel cannot meet the demand of people.People are badly in need of using multi-channel video simultaneously, obtain larger visual angle.
Existing multi-channel video processing system is all confined to a certain professional domain to a great extent, cannot be extended to other multi-channel video processing field very easily.The video monitoring system of such as multichannel and the difference of overall view monitoring system are only the stitching algorithm of multiway images, but all functions of these two kinds of systems all need all again to develop: the collection comprising video, transmission, and encoding and decoding, display, stores etc.Urgently develop a kind of video image system framework applications in the processing system for video of multichannel.
Summary of the invention
The object of the invention is to overcome the deficiencies in the prior art, provide a kind of cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture
The present invention is achieved by the following technical solutions: based on the cross-platform multichannel Real-time Video Processing System framework of pipelined architecture, comprise system layer, level of abstraction and storehouse layer, described storehouse layer comprises cross-platform assembly c++11, ffmpeg, sdl and opengl;
Described system layer comprises control module, Computer Vision module and video image and preserves module; Described level of abstraction comprises video resource acquisition module, Video decoding module, audio video synchronization module and image display;
Video acquisition module gathers external video data, starts control module, and source obtaining module reads the image bag data of collection automatically frequently, and video resource acquisition module is read the image bag gathered and is decoded as corresponding picture frame by Computer Vision module; Control module obtains synchronous sequence of frames of video from audio video synchronization module, then is transmitted in Computer Vision resume module; Control module obtains the image/video after process for image display display, and startup video image is preserved module and preserved video image simultaneously.
As the further optimization of such scheme, described control module is for responding user's request, and Computer Vision module, described video resource acquisition module and described Video decoding module described in cooperation control.
As the further optimization of such scheme, described level of abstraction also comprises image scaling processing module, for the control command of response module, carries out convergent-divergent process to the video image that video resource acquisition module obtains.
As the further optimization of such scheme, described audio video synchronization module realizes based on the synchronized algorithm of frame of video corresponding time, and each frame data in video source are preserved in the synchronous chained list queue of corresponding secondary.
The present invention has the following advantages compared to existing technology: the invention provides a kind of cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture, have real-time, platform-neutral, video source independence, the advantages such as video format independence.But the invention provides a kind of cross-platform multichannel Real-time Video Processing System frame based on pipelined architecture by abstract for multi-channel video processing system be some separate co-operating subsystems, do not interdepend between each subsystem, and can replace and combine to realize the demand of different application scene flexibly.Improve the durability of system component, flexibility, extensibility and stability.
Accompanying drawing explanation
Fig. 1 is the system architecture diagram of the cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture of the present invention.
Fig. 2 is the module interaction figure of the cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture of the present invention.
Fig. 3 is the realization flow interaction figure of the cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture of the present invention.
Fig. 4 is the synchronous process schematic diagram of the preferred embodiment of the cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture of the present invention.
Embodiment
Elaborate to embodiments of the invention below, the present embodiment is implemented under premised on technical solution of the present invention, give detailed execution mode and concrete operating process, but protection scope of the present invention is not limited to following embodiment.
See Fig. 1, it is the system architecture diagram of the preferred embodiments of the present invention.Based on the cross-platform multichannel Real-time Video Processing System framework of pipelined architecture, comprise system layer, level of abstraction and storehouse layer.Storehouse layer comprises cross-platform assembly c++11, FFmpeg, sdl SDL and OpenGL.System layer comprises control module, Computer Vision module and video image and preserves module.Level of abstraction comprises video resource acquisition module, Video decoding module, audio video synchronization module and image display.
Storehouse layer encapsulates by level of abstraction, the dependence of decoupled system layer and storehouse layer.Image display encapsulation sdl storehouse, decoupled system layer associates with sdl storehouse.The function of system layer combination level of abstraction, the function of collaborate completion system.System layer adopts the framework of MVC, and wherein control module is the control centre in MVC.
FFmpeg (Fast Forward mpeg): FFmpeg is a set of record, converted digital audio, video of can being used for, and can be translated into the storehouse of increasing income of stream.Adopt LGPL or GPL licence.It provide recording, conversion and the total solution of fluidisation audio frequency and video.
SDL (Simple DirectMedia Layer): the cross-platform multimedia development storehouse being a set of open source code, uses C language to be write as.SDL provides the function that several control image, sound, import and export, as long as allow developer with identical or similar code development across multiple platform, as the application software of Linux, Windows, Mac OS X etc.
An OpenGL (Open Graphics Library): be define a graphic package interface across the specialty of programming language, cross-platform DLL (dynamic link library) specification.It is for 3-D view, two dimension also can, be one powerful, call underlying graphics storehouse easily.
C++11 (ISO/IEC 14882:2011-Information technology--Programming languages--C++): replacement C++ standard second edition ISO/IEC14882:2003-Programming languages--C++ is become C Plus Plus new standard by it.C++11 contains the new function of kernel language, and expands C++ standard library, and adds most C++Technical Report 1 program library.
See Fig. 2, it is the module interaction figure of the cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture of the present invention.Control module MultiPlayer control synchronization module SyncQueue, interior processing module Player, image scaling processing module Scaler, Computer Vision module I mageProcessor, video image preserve module Recorder, image display SDL for showing the video image through control module MultiPlayer manipulation.
Interior processing module Player composite video source obtaining module PlayerSource and Video decoding module Stream obtains image decoding from video source, and processes disconnection and reconnect.Each video source is the manipulation object of an interior processing module Player.
Synchronization module SyncQueue is synchronous and give Computer Vision module I mageProcessor process by synchronized algorithm by the frame of video of multiple interior processing module Player decoding.Computer Vision module I mageProcessor adopts merging algorithm for images to be merged by the video data of multiple interior processing module Player.Synchronization module SyncQueue realizes based on time synchronized queue TimeSyncQueue, and the synchronized algorithm based on video frame time realizes.
Video resource acquisition module PlayerSource obtains the video source FFmpegPlayerSource of storehouse layer FFmpeg.Video decoding module Stream realizes based on the video flowing FFmpegStream of storehouse layer FFmpeg.
Object video realizes audio video synchronization module PacketQueue based on the general type isochronous queue that semaphore realizes and synchronously realizes.Object video comprises image bag MyAVPacket, picture frame MyAVFrame and picture MyAVPicture.Wherein, image bag MyAVPacket is the video data of memory encoding, and picture frame MyAVFrame stores a decoded frame video image, and picture MyAVPicture is the image after storing convergent-divergent
See the realization flow interaction figure that Fig. 3 is the cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture.Video acquisition module gathers external video data, starts control module MultiPlayer, and interior processing module Player starts internal thread reads collection automatically image bag MyAVPacket data from frequency source obtaining module PlayerSource; Video resource acquisition module PlayerSource is read the image bag MyAVPacket gathered and is decoded as corresponding picture frame MyAVFrame by the internal thread of interior processing module Player; Control module MultiPlayer obtains synchronous sequence of frames of video from the time synchronized queue TimeSyncQueue audio video synchronization module, then is transmitted in Computer Vision module I mageProcessor process; Control module MultiPlayer obtains the image/video after process for image display display, and startup video image is preserved module Recorder and preserved video image simultaneously.
Based on the general type isochronous queue of Semaphore Mechanism
Pipelined architecture unavoidably needs the result using queue buffer memory upper level streamline.Audio video synchronization module PacketQueue is designed to the general type isochronous queue of thread-safe, and uses semaphore to realize the synchronous of cross-thread.Audio video synchronization module PacketQueue mainly contains Put () and Get () operation, realizes false code as follows:
PacketQueue<class T>::Put(shared_ptr<T>item)：
1.Lock(mutex)；
2.PushQueue(queue,item)；
3.if(QueueFull(queue))PopQueue(queue)；
4.Notify(Semaphore)；
5.Unlock(mutex)；
PacketQueue<class T>::Get(int block,int waitMils,int pop)：
1.Lock(mutex)；
2.shared_ptr<T>item；
3.while(True):
4.if(HasItem(queue)):
5.item＝GetItem(queue)；
6.if(pop)PopQueue(queue)；
7.break；
8.else if(block):
9.Wait(Semaphore,waitMils)；
10.else
11.break；
12.Unlock(mutex)；
13.return item；
The synchronized algorithm of the time synchronized queue TimeSyncQueue of audio video synchronization module PacketQueue is one group of sequence of frames of video that in the frame of video in order to take out different video source, the time is the most close.0th frame of Player0 is as shown in Figure 4 at 80ms place, the frame of video of Player1 is at 70ms place, the frame of video 0 of Player2 is at 30ms place, frame of video 1 is at 70ms place, then synchronized algorithm can skip the 0th frame of Player2, by the 0th frame of Player0, the 0th frame of Player1 and the 1st frame of Player2 take out.
Synchronized algorithm can choose one for masterPlayer in Player0-PlayerN, and can using the initial time of masterPlayer first frame as lock in time.Each frame in other slavePlayer can contrast with the initial time of masterPlayer, selects an immediate frame.
The false code that synchronized algorithm realizes is as follows:
system_clock::time_point
TimeSyncQueue::Get(DYSyncQueueResultList&result)
1.masterItem＝GetItem(masterPlayer)；
2.if(NULL＝＝masterItem&&NeedChangeMaster())ChangeMaster(&masterPlayer)；
3.return；
4.for(slavePlayer in slavePlayerList):
5.slaveItem＝NULL；
6.do:
7.slaveItem＝GetItem(slavePlayer)；
8.if(NULL＝＝slaveItem)break；
9.slaveCache＝slaveItem//cache
10.if(CheckDuration(slaveItem,masterItem))break；
11.while(true)
12.if(slaveItem＝＝NULL)slaveItem＝slaveCache；//use lastcache instead
13.ClearCacheIfNecessary(slaveCache)；
The invention provides a kind of cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture, there is real-time, platform-neutral, video source independence, the advantages such as video format independence.But the invention provides a kind of cross-platform multichannel Real-time Video Processing System frame based on pipelined architecture by abstract for multi-channel video processing system be some separate co-operating subsystems, do not interdepend between each subsystem, and can replace and combine to realize the demand of different application scene flexibly.Improve the durability of system component, flexibility, extensibility and stability.
The invention provides a kind of cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture to be mainly used in vehicle-mounted 360 degree of panoramic parking systems.Also can be applied in other multi-channel video processing system very easily, as the video monitoring system etc. of multichannel simultaneously.360 degree of panoramic parking systems generally can all around install 4 bugeye lenses at vehicle body, the 360 panorama vertical views that final driving one secondary vehicle's surroundings is seamless after image procossing.While display panoramic picture, also can show either party single-view.
The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, all any amendments done within the spirit and principles in the present invention, equivalent replacement and improvement etc., all should be included within protection scope of the present invention.
Claims (4)
1., based on the cross-platform multichannel Real-time Video Processing System framework of pipelined architecture, it is characterized in that, comprising: system layer, level of abstraction and storehouse layer, described storehouse layer comprises cross-platform assembly c++11, ffmpeg, sdl and opengl,
Described system layer comprises control module, Computer Vision module and video image and preserves module; Described level of abstraction comprises video resource acquisition module, Video decoding module, audio video synchronization module and image display;
Video acquisition module gathers external video data, starts control module, and source obtaining module reads the image bag data of collection automatically frequently, and video resource acquisition module is read the image bag gathered and is decoded as corresponding picture frame by Computer Vision module; Control module obtains synchronous sequence of frames of video from audio video synchronization module, then is transmitted in Computer Vision resume module; Control module obtains the image/video after process for image display display, and startup video image is preserved module and preserved video image simultaneously.
2. the cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture according to claim 1, it is characterized in that: described control module is for responding user's request, and Computer Vision module, described video resource acquisition module and described Video decoding module described in cooperation control.
3. the cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture according to claim 1, it is characterized in that: described level of abstraction also comprises image scaling processing module, for the control command of response module, convergent-divergent process is carried out to the video image that video resource acquisition module obtains.
4. the cross-platform multichannel Real-time Video Processing System framework based on pipelined architecture according to claim 1, it is characterized in that: described audio video synchronization module realizes based on the synchronized algorithm of time corresponding to vedio data frame, each frame data in video source are preserved in the synchronous chained list queue of corresponding secondary.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510197275.3A CN104796670A (en) | 2015-04-23 | 2015-04-23 | Cross-platform multiway realtime video processing system architecture based on flow line architecture |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510197275.3A CN104796670A (en) | 2015-04-23 | 2015-04-23 | Cross-platform multiway realtime video processing system architecture based on flow line architecture |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104796670A true CN104796670A (en) | 2015-07-22 |
Family
ID=53561149
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510197275.3A Pending CN104796670A (en) | 2015-04-23 | 2015-04-23 | Cross-platform multiway realtime video processing system architecture based on flow line architecture |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104796670A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112788282A (en) * | 2019-11-08 | 2021-05-11 | 株洲中车时代电气股份有限公司 | Video information acquisition method and application thereof |
| CN115514940A (en) * | 2022-11-22 | 2022-12-23 | 天地伟业技术有限公司 | Method for building multi-view video monitoring system and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101641690A (en) * | 2007-01-22 | 2010-02-03 | E3网络股份有限公司 | Be used to generate the method and the middleware platform engine thereof of cross platform program |
| CN101814024A (en) * | 2009-10-23 | 2010-08-25 | 深圳创维－Rgb电子有限公司 | Framework and tool for designing cross-platform 3D interface |
| CN102158762A (en) * | 2011-03-15 | 2011-08-17 | 深圳市酷开网络科技有限公司 | Method and system for achieving television user interface based on SDL (System Descriptive Language) |
| CN103888771A (en) * | 2013-12-30 | 2014-06-25 | 中山大学深圳研究院 | Parallel video image processing method based on GPGPU technology |
| CN104410807A (en) * | 2014-11-24 | 2015-03-11 | 深圳市华宝电子科技有限公司 | Method and device for synchronously replaying multi-channel video |
-
2015
- 2015-04-23 CN CN201510197275.3A patent/CN104796670A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101641690A (en) * | 2007-01-22 | 2010-02-03 | E3网络股份有限公司 | Be used to generate the method and the middleware platform engine thereof of cross platform program |
| CN101814024A (en) * | 2009-10-23 | 2010-08-25 | 深圳创维－Rgb电子有限公司 | Framework and tool for designing cross-platform 3D interface |
| CN102158762A (en) * | 2011-03-15 | 2011-08-17 | 深圳市酷开网络科技有限公司 | Method and system for achieving television user interface based on SDL (System Descriptive Language) |
| CN103888771A (en) * | 2013-12-30 | 2014-06-25 | 中山大学深圳研究院 | Parallel video image processing method based on GPGPU technology |
| CN104410807A (en) * | 2014-11-24 | 2015-03-11 | 深圳市华宝电子科技有限公司 | Method and device for synchronously replaying multi-channel video |
Non-Patent Citations (3)
| Title |
|---|
| 尚海忠，朱培彦，王霞等: "操作系统抽象层-一种支持跨平台的新技术", 《计算机工程》 * |
| 李科，李璐，兰时勇: "基于FFmpeg和SDL实现多路实时流变换及播放", 《计算机技术与发展》 * |
| 蒋鹏: "基于SDL库的跨平台多媒体应用", 《福建电脑》 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112788282A (en) * | 2019-11-08 | 2021-05-11 | 株洲中车时代电气股份有限公司 | Video information acquisition method and application thereof |
| CN115514940A (en) * | 2022-11-22 | 2022-12-23 | 天地伟业技术有限公司 | Method for building multi-view video monitoring system and electronic equipment |
| WO2024108703A1 (en) * | 2022-11-22 | 2024-05-30 | 天地伟业技术有限公司 | Multi-view video monitoring system building method and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11750787B2 (en) | Adaptive resolution of point cloud and viewpoint prediction for video streaming in computing environments | |
| CN111213183A (en) | Method and device for rendering three-dimensional content | |
| WO2018048287A1 (en) | Image processing method and device for projecting image of virtual reality content | |
| EP1784021B1 (en) | Video processing with multiple graphics processing units | |
| US9001229B2 (en) | Multiple sensor input data synthesis | |
| US10148875B1 (en) | Method and system for interfacing multiple channels of panoramic videos with a high-definition port of a processor | |
| CN110073662A (en) | The suggestion viewport of panoramic video indicates | |
| WO2014052437A1 (en) | Encoding images using a 3d mesh of polygons and corresponding textures | |
| WO2021018070A1 (en) | Image display method and electronic device | |
| CN111064906A (en) | Domestic processor and domestic FPGA multi-path 4K high-definition video comprehensive display method | |
| CN101877767A (en) | Method and system for generating three-dimensional panoramic continuous video through six-channel video source | |
| WO2018103384A1 (en) | Method, device and system for playing 360 degree panoramic video | |
| CN209028563U (en) | A kind of VR all-in-one machine for cooperating PC to use | |
| US20200366962A1 (en) | Bitstream merger and extractor | |
| TW201810189A (en) | File format for indication of video content | |
| CN106530312B (en) | One kind being based on low-power-consumption embedded system real-time image scratching method | |
| CN103037169A (en) | Picture split joint combination method of embedded hard disk video | |
| CN104796670A (en) | Cross-platform multiway realtime video processing system architecture based on flow line architecture | |
| CN107580228B (en) | Monitoring video processing method, device and equipment | |
| CN104581075A (en) | Panoramic video processing system and method based on heterogeneous platform | |
| CN105163128B (en) | A kind of screen picture acquisition methods for accelerating image to change parallel using GPU | |
| WO2023280266A1 (en) | Fisheye image compression method, fisheye video stream compression method and panoramic video generation method | |
| Kim et al. | A real-time MPEG encoder using a programmable processor | |
| US7298425B2 (en) | Method for assisting video compression in a computer system | |
| JP2014041455A (en) | Image processing device, image processing method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150722 |
|
| RJ01 | Rejection of invention patent application after publication |