CN107197318A - A kind of real-time, freedom viewpoint live broadcasting method shot based on multi-cam light field - Google Patents
A kind of real-time, freedom viewpoint live broadcasting method shot based on multi-cam light field Download PDFInfo
- Publication number
- CN107197318A CN107197318A CN201710464275.4A CN201710464275A CN107197318A CN 107197318 A CN107197318 A CN 107197318A CN 201710464275 A CN201710464275 A CN 201710464275A CN 107197318 A CN107197318 A CN 107197318A
- Authority
- CN
- China
- Prior art keywords
- user
- viewpoint
- data
- camera
- light field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/65—Transmission of management data between client and server
- H04N21/658—Transmission by the client directed to the server
- H04N21/6587—Control parameters, e.g. trick play commands, viewpoint selection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Image Processing (AREA)
Abstract
The present invention is a kind of real-time, freedom viewpoint live broadcasting method shot based on multi-cam light field, the live and recorded broadcast of the fixation stage region performance by any angle, any viewpoint of user's control can be achieved in the present invention, relative to panoramic video, it is an advantage of the invention that user's free movement can be made, visual angle and viewpoint are changed；Live benefit relative to three-dimensionalreconstruction is quick, efficient, the not detailed information loss without modeling, closer to photo level quality；Relative to light field transmission plan, because the data that the viewpoint that the present invention is user is dynamically selected the packet of generation user's visual point image are transmitted, it can support to greatly reduce volume of transmitted data to the packet of large-scale concurrent user viewpoint, overcome free view-point mobile virtual practical application in the data transmission bottle neck of internet large-scale application；In addition, output image of the invention is synthesized by depth information and neighbouring camera, can greatly reduce the quantity of camera.
Description
Technical field
Invention is related to virtual reality interactive application field, extensive straight in real time especially for three-dimensional scenic in virtual reality
Broadcasting device and algorithm.
Background technology
Tradition is recorded：Existing video frequency program is mainly played by flat-panel screens, and 3D is played by wearing 3D glasses
Video can realize 3D visual effects, but its visual angle and broadcasting content are set by program making side, and user can only passively receive.
360 degree of panoramas are recorded：With the popularization of virtual reality technology, virtual reality video frequency program is also gradually being popularized, and is used
Family is by wearing VR head-mounted displays, by by Softcam consistent method synchronous with the angle of VR head-mounted displays, can
To realize the immersion video tastes for being similar to ring curtain effect, but this method can only provide the user with rotation in 1 viewpoint
The free degree of viewing, user can not move back and forth in virtual scene.
Either common video frequency program or VR video frequency programs, can not be supplied to facing for the comprehensive immersion of user
Field experience, ordinary video program can only allow user to be watched by screen window, and VR video frequency programs are also merely able to allow user solid
Fixed viewpoint or the circuit selected by photographer are watched, and this has spy to competitive sports, variety show, concert, news conference etc.
Surely the program in performance region is by great content loss, and scene can not comprehensively be obtained by watching program by these existing modes
Information.
The content of the invention
It is an object of the invention to provide it is a kind of based on multi-cam light field shoot real-time, freedom viewpoint live broadcasting method, with
The problem of solving to propose in above-mentioned background technology.
To achieve the above object, the present invention provides following technical scheme：
A kind of real-time, freedom viewpoint live broadcasting method shot based on multi-cam light field, is comprised the steps of：
A, camera acquisition vision signal, and it is transferred to processing equipment；
B, processing equipment generation depth data, calculate each pixel and the distance of camera；
C, the polygon constituted by camera or triangle pair camera are grouped, and the video data stream of these packets is carried out
Merge compression；
D, the observation position according to user, program automatically select corresponding compression video and depth data, and are assembled into data flow
By network transmission to user；
E, subscription client are decompressed to data；
F, the image of the view data according to several angles and viewpoint and visual angle where depth data synthesis user simultaneously render display
Out.
It is used as the further technical scheme of the present invention：The step 2 also has second of mode of operation, i.e., by client
Reason.
Compared with prior art, the beneficial effects of the invention are as follows：The present invention can be achieved by user's control any angle, appoint
The live and recorded broadcast of the fixation stage region performance for viewpoint of anticipating, relative to panoramic video, it is an advantage of the invention that user can be made
Free movement, changes visual angle and viewpoint；Live benefit relative to three-dimensionalreconstruction is without modeling, quickly, efficiently, not
Detailed information is lost, closer to photo level quality；Relative to light field transmission plan because the present invention be the viewpoint of user dynamically
The data of the packet of selection generation user's visual point image are transmitted, and can support greatly to subtract the packet of large-scale concurrent user viewpoint
Few volume of transmitted data, overcomes free view-point mobile virtual practical application in the data transmission bottle neck of internet large-scale application；Remove
Outside this, output image of the invention is synthesized by depth information and neighbouring camera, can greatly reduce the quantity of camera.
Brief description of the drawings
Fig. 1 is general illustration of the invention；
Fig. 2 is packet schematic diagram of the invention；
Fig. 3 is hardware architecture diagram of the invention；
Fig. 4 is flow chart of the invention.
Embodiment
The technical scheme in the embodiment of the present invention will be clearly and completely described below, it is clear that described implementation
Example only a part of embodiment of the invention, rather than whole embodiments.Based on the embodiment in the present invention, this area is common
The every other embodiment that technical staff is obtained under the premise of creative work is not made, belongs to the model that the present invention is protected
Enclose.
Refering to Fig. 1-4；In the embodiment of the present invention, a kind of live side of real-time, freedom viewpoint shot based on multi-cam light field
Method, as shown in figure, the present invention, around stage or performance region, video are shot from multiple angles simultaneously using multiple cameras
Data, for the user watched in any angle, can find a triangle being made up of several cameras or polygon area
Domain covers the visual angle of user.
As shown in Figure 2,1,3, No. 4 cameras can cover the user of the Delta Region surrounded by 1,3,4 three point, 1,2,4
Number camera can cover the user of the Delta Region surrounded by 1,2,4 three point, by that analogy.
As shown in Figure 3, the video data of every camera will be concentrated to video distribution servers, and these data pass through interconnection
Net is transferred to user.
As shown in Figure IV, flow of the invention can be summarized as 7 steps：
The first step：Camera acquisition vision signal, and it is transferred to processing equipment；
Second step：Processing equipment generates depth data, calculates each pixel and the distance of camera；This step can also be such as the mould of figure four
Shown in formula two, by client process；
3rd step：The polygon or triangle pair camera constituted by camera is grouped, and the video data that these are grouped
Stream merges compression, and figure two describes a kind of packet mode by triangle overlay area, and such as 1,3, No. 4 cameras are closed
And data flow A is compressed into, 1,2, No. 4 camera merges and is compressed into data stream B；
4th step and the 5th step：According to the observation position of user, program automatically selects corresponding compression video and depth data, and
Data flow is assembled into by network transmission to user, in figure two, for the user in 1,3,4 field of view, A numbers will be received
According to stream, in the user of 1,2,4 field of view, B data stream will be received；When user from a-quadrant is moved to B regions, data flow
Need while switching over；
In the network transmission stage, for internet large-scale concurrent user, user is grouped by the present invention according to its viewpoint, together
One group of user is using same group of compressed data, to reduce bandwidth and amount of calculation needed for the transmission of data interconnection net.
6th step：Subscription client is decompressed to data；
7th step：The image of user's place viewpoint and visual angle is synthesized according to the view data of several angles and depth data and is rendered
Show.
The better embodiment to the present invention is illustrated above, but the invention is not limited to the implementation
Example, those skilled in the art can also make a variety of equivalent modifications or replace on the premise of without prejudice to spirit of the invention
Change, these equivalent modifications or replacement are all contained in the application claim limited range.
Claims (2)
1. a kind of real-time, freedom viewpoint live broadcasting method shot based on multi-cam light field, it is characterised in that comprise the steps of：
A, camera acquisition vision signal, and it is transferred to processing equipment；
B, processing equipment generation depth data, calculate each pixel and the distance of camera；
C, the polygon constituted by camera or triangle pair camera are grouped, and the video data stream of these packets is carried out
Merge compression；
D, the observation position according to user, program automatically select corresponding compression video and depth data, and are assembled into data flow
By network transmission to user；
E, subscription client are decompressed to data；
F, the image of the view data according to several angles and viewpoint and visual angle where depth data synthesis user simultaneously render display
Out.
2. a kind of real-time, freedom viewpoint live broadcasting method shot based on multi-cam light field according to claim 5, it is special
Levy and be, the step 2 also has second of mode of operation, i.e., by client process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710464275.4A CN107197318A (en) | 2017-06-19 | 2017-06-19 | A kind of real-time, freedom viewpoint live broadcasting method shot based on multi-cam light field |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710464275.4A CN107197318A (en) | 2017-06-19 | 2017-06-19 | A kind of real-time, freedom viewpoint live broadcasting method shot based on multi-cam light field |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107197318A true CN107197318A (en) | 2017-09-22 |
Family
ID=59878606
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710464275.4A Pending CN107197318A (en) | 2017-06-19 | 2017-06-19 | A kind of real-time, freedom viewpoint live broadcasting method shot based on multi-cam light field |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107197318A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107918948A (en) * | 2017-11-02 | 2018-04-17 | 深圳市自由视像科技有限公司 | 4D Video Rendering methods |
| CN108418832A (en) * | 2018-03-26 | 2018-08-17 | 深圳市酷开网络科技有限公司 | A kind of virtual reality shopping guide method, system and storage medium |
| CN110198457A (en) * | 2018-02-26 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Video broadcasting method and its equipment, system, storage medium, terminal, server |
| CN110430416A (en) * | 2019-07-17 | 2019-11-08 | 清华大学 | Free view-point image generating method and device |
| CN110944222A (en) * | 2018-09-21 | 2020-03-31 | 上海交通大学 | Method and system for immersive media content as user moves |
| CN111931071A (en) * | 2020-10-09 | 2020-11-13 | 北京瞰瞰科技有限公司 | Video data pushing method and device |
| CN114638771A (en) * | 2022-03-11 | 2022-06-17 | 北京拙河科技有限公司 | Video fusion method and system based on hybrid model |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102215380A (en) * | 2010-04-09 | 2011-10-12 | 霍尼韦尔国际公司 | Systems and methods to group and browse cameras in a large scale surveillance system |
| US20110293236A1 (en) * | 2010-05-28 | 2011-12-01 | Hon Hai Precision Industry Co., Ltd. | Digital video recorder and managing method |
| CN105791881A (en) * | 2016-03-15 | 2016-07-20 | 深圳市望尘科技有限公司 | Optical-field-camera-based realization method for three-dimensional scene recording and broadcasting |
| CN105939481A (en) * | 2016-05-12 | 2016-09-14 | 深圳市望尘科技有限公司 | Interactive three-dimensional virtual reality video program recorded broadcast and live broadcast method |
| CN106060513A (en) * | 2016-06-29 | 2016-10-26 | 深圳市优象计算技术有限公司 | Code stream caching mechanism for cylindrical surface stereoscopic panoramic video network playing |
-
2017
- 2017-06-19 CN CN201710464275.4A patent/CN107197318A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102215380A (en) * | 2010-04-09 | 2011-10-12 | 霍尼韦尔国际公司 | Systems and methods to group and browse cameras in a large scale surveillance system |
| US20110293236A1 (en) * | 2010-05-28 | 2011-12-01 | Hon Hai Precision Industry Co., Ltd. | Digital video recorder and managing method |
| CN105791881A (en) * | 2016-03-15 | 2016-07-20 | 深圳市望尘科技有限公司 | Optical-field-camera-based realization method for three-dimensional scene recording and broadcasting |
| CN105939481A (en) * | 2016-05-12 | 2016-09-14 | 深圳市望尘科技有限公司 | Interactive three-dimensional virtual reality video program recorded broadcast and live broadcast method |
| CN106060513A (en) * | 2016-06-29 | 2016-10-26 | 深圳市优象计算技术有限公司 | Code stream caching mechanism for cylindrical surface stereoscopic panoramic video network playing |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107918948A (en) * | 2017-11-02 | 2018-04-17 | 深圳市自由视像科技有限公司 | 4D Video Rendering methods |
| CN110198457A (en) * | 2018-02-26 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Video broadcasting method and its equipment, system, storage medium, terminal, server |
| CN108418832A (en) * | 2018-03-26 | 2018-08-17 | 深圳市酷开网络科技有限公司 | A kind of virtual reality shopping guide method, system and storage medium |
| CN110944222A (en) * | 2018-09-21 | 2020-03-31 | 上海交通大学 | Method and system for immersive media content as user moves |
| CN110944222B (en) * | 2018-09-21 | 2021-02-12 | 上海交通大学 | Method and system for immersive media content as user moves |
| CN110430416A (en) * | 2019-07-17 | 2019-11-08 | 清华大学 | Free view-point image generating method and device |
| CN110430416B (en) * | 2019-07-17 | 2020-12-08 | 清华大学 | Free viewpoint image generation method and device |
| CN111931071A (en) * | 2020-10-09 | 2020-11-13 | 北京瞰瞰科技有限公司 | Video data pushing method and device |
| CN114638771A (en) * | 2022-03-11 | 2022-06-17 | 北京拙河科技有限公司 | Video fusion method and system based on hybrid model |
| CN114638771B (en) * | 2022-03-11 | 2022-11-29 | 北京拙河科技有限公司 | Video fusion method and system based on hybrid model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107197318A (en) | A kind of real-time, freedom viewpoint live broadcasting method shot based on multi-cam light field | |
| WO2021179783A1 (en) | Free viewpoint-based video live broadcast processing method, device, system, chip and medium | |
| US10438633B2 (en) | Method and system for low cost television production | |
| CN106165415B (en) | Stereoscopic viewing | |
| US8228327B2 (en) | Non-linear depth rendering of stereoscopic animated images | |
| US20150358539A1 (en) | Mobile Virtual Reality Camera, Method, And System | |
| CN105939481A (en) | Interactive three-dimensional virtual reality video program recorded broadcast and live broadcast method | |
| CN106878764A (en) | A kind of live broadcasting method of virtual reality, system and application thereof | |
| US20060244831A1 (en) | System and method for supplying and receiving a custom image | |
| CN102340690A (en) | Interactive television program system and realization method | |
| CN104335243A (en) | Processing panoramic pictures | |
| CN106060513B (en) | A kind of code stream caching method for cylinder three-dimensional panoramic video netcast | |
| CN108900857A (en) | A kind of multi-visual angle video stream treating method and apparatus | |
| CN105847778B (en) | 360 ° of multiple views 3D holographic videos acquisition methods, equipment and implementation method | |
| CN106657719A (en) | Intelligent virtual studio system | |
| CN108134928A (en) | VR display methods and device | |
| CN108153412A (en) | LAN VR live broadcasting methods and system | |
| Domański et al. | Demonstration of a simple free viewpoint television system | |
| CN100486140C (en) | Three-D vedio monitoring method and system based on internet | |
| EP2852149A1 (en) | Method and apparatus for generation, processing and delivery of 3D video | |
| US11930290B2 (en) | Panoramic picture in picture video | |
| CN107197316A (en) | Panorama live broadcast system and method | |
| US11792380B2 (en) | Video transmission method, video processing device, and video generating system for virtual reality | |
| RU206248U1 (en) | Multi-screen panoramic TV | |
| Naemura et al. | Multiresolution stereoscopic immersive communication using a set of four cameras |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170922 |