CN109729368A - The method and system of VR live video processing - Google Patents
The method and system of VR live video processing Download PDFInfo
- Publication number
- CN109729368A CN109729368A CN201811366415.5A CN201811366415A CN109729368A CN 109729368 A CN109729368 A CN 109729368A CN 201811366415 A CN201811366415 A CN 201811366415A CN 109729368 A CN109729368 A CN 109729368A
- Authority
- CN
- China
- Prior art keywords
- frame
- picture
- background
- background frame
- video flowing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses the methods of VR live video processing, and method: collection terminal acquires background frame and motion picture, detect moving object；Moving object region is determined for every frame motion picture, first area corresponding with region in background frame, intercept the first picture of region；By background frame, the first picture, the first picture, corresponding first area reaches server end in background frame；The data received are issued client by server end；Client replicates according to background frame for every the first picture of frame and generates the first background frame, the first area that corresponding first picture affixes to the first background frame is formed complete picture, all complete pictures form the first video flowing；Rendering output is carried out to the first video flowing.The invention also discloses a kind of systems.Method and system of the invention alleviates collection terminal to server end uploading bandwidth pressure and server end to client and issues bandwidth pressure under the premise of not influencing the clarity of VR live video.
Description
Technical field
The present invention relates to the method and systems that communication technique field more particularly to a kind of VR live video are handled.
Background technique
Network direct broadcasting is very hot at present, it is stated that the just prosperous live streaming platform of public attention has reached 116.It is emerging with VR
It rises, VR live video also can slowly prevail.This 360 degree of panoramic effects of VR can bring completely new experience to viewer, must
It is well-established greatly by general sieve that a kind of new direct-seeding will be become.
VR live video needs to complete 360 ° of panorama material collection, and for now, the acquisition of VR panoramic video only has
It is acquired by multiple wide-angle lens.Then a complete panoramic video is spliced by computer graphics algorithm.
VR video uses spherical projection mode at present, only watches in a portion picture when being watched by VR terminal
Hold.Therefore under the premise of image sharpness is identical, the resolution ratio of the resolution ratio tradition live video of VR live video is bigger.
Under conditions of i.e. real-time broadcasting pictures clarity same video, VR live video needs bigger network to pass than traditional live video
Defeated bandwidth brings very big pressure to current network bandwidth.
Summary of the invention
It is bigger compared to traditional live video needs it is a primary object of the present invention to solve VR live video in the prior art
Network transmission bandwidth, the technical issues of bringing very big pressure to current network bandwidth.
To achieve the above object, the present invention provides a kind of method of VR live video processing, the VR live video processing
Method include:
Collection terminal acquires background frame and subsequent motion picture in real time, and the background frame and the motion picture are formed
Video flowing detects the moving object in video flowing；
For every frame motion picture, determine the region of moving object, in the background frame with the region
Corresponding first area intercepts the first picture of the region；
In real time by the first picture described in the first picture described in the background frame, every frame and every frame in the background painting
The corresponding first area is uploaded to server end in face；
The server end in real time will be first described in the received background frame of institute, first picture and every frame
It is handed down to client in picture corresponding first area in the background frame；
The client replicates according to the background frame for the first picture described in every frame and generates first background painting of frame
Face affixes to corresponding first picture in first background frame in corresponding first area, forms a frame complete picture,
All complete pictures are formed into the first video flowing；
Rendering output is carried out to first video flowing.
Preferably, the collection terminal detects the fortune in the video flowing using optical flow method, frame difference method or background subtraction
Animal body.
Preferably, first picture of each frame carries timestamp of the corresponding frame motion picture in the video flowing；
All complete pictures are formed the first video flowing and specifically included by the client:
The timestamp that the client is carried according to corresponding first picture of every frame complete picture, by all complete pictures
Form first video flowing.
In addition, to achieve the above object, the present invention also provides a kind of system of VR live video processing, the VR live streaming view
Frequently the system of processing includes:
Collection terminal, for acquiring background frame and subsequent motion picture, the background frame and the movement picture in real time
Face forms video flowing, detects the moving object in video flowing；For every frame motion picture, determine moving object region,
First area corresponding with the region in the background frame, intercepts the first picture of the region；And it is real
When the first picture described in the first picture described in the background frame, every frame and every frame is corresponding in the background frame
The first area is uploaded to server end；
The server end, in real time will be described in the received background frame of institute, first picture and every frame
It is handed down to client in first picture corresponding first area in the background frame；
The client, for being replicated according to the background frame and generating a frame first for the first picture described in every frame
Background frame affixes to corresponding first picture in first background frame in corresponding first area, and it is complete to form a frame
All complete pictures are formed the first video flowing by whole picture；Rendering output is carried out to first video flowing.
Preferably, the collection terminal detects the fortune in the video flowing using optical flow method, frame difference method or background subtraction
Animal body.
Preferably, first picture of each frame carries timestamp of the corresponding frame motion picture in the video flowing；
The client will own specifically for the timestamp carried according to corresponding first picture of every frame complete picture
Complete picture form first video flowing.
The method and system of VR live video provided by the present invention processing, acquired in real time by collection terminal background frame and
Subsequent motion picture, the background frame and the motion picture form video flowing, detect the moving object in video flowing；Needle
To every frame motion picture, the region of moving object is determined, corresponding with the region first in the background frame
Region intercepts the first picture of the region；In real time by the first picture described in the background frame, every frame and every frame
First picture corresponding first area in the background frame is uploaded to server end；The server end is real-time
First picture described in the received background frame of institute, first picture and every frame is corresponding in the background frame
The first area be handed down to client；The client is multiple according to the background frame for the first picture described in every frame
System generates first background frame of frame, and corresponding first picture is affixed to corresponding first area in first background frame
In, a frame complete picture is formed, all complete pictures are formed into the first video flowing；Rendering output is carried out to first video flowing
Mode greatly reduce collection terminal to service under the premise of not influencing the clarity of client terminal playing VR live video
The data volume and reduce the data volume that server end is issued to client that device end uploads；To alleviate from collection terminal to
The uploading bandwidth pressure of server end, and alleviate and issue bandwidth pressure from server end to client.
Detailed description of the invention
Fig. 1 is the flow diagram for one embodiment of method that VR live video of the invention is handled；
Fig. 2 is the structural schematic diagram for one embodiment of system that VR live video of the invention is handled.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
Below in conjunction with Figure of description, preferred embodiment of the present invention will be described, it should be understood that described herein
Preferred embodiment only for the purpose of illustrating and explaining the present invention and is not intended to limit the present invention, and in the absence of conflict, this hair
The feature in embodiment and embodiment in bright can be combined with each other.
The present invention provides a kind of method of VR live video processing.Referring to Fig.1, Fig. 1 is at VR live video of the invention
The flow diagram of one embodiment of method of reason.In one embodiment, the method for the VR live video processing includes:
Step S10: collection terminal acquires background frame and subsequent motion picture, the background frame and the movement in real time
Picture forms video flowing, detects the moving object in video flowing.
Collection terminal described in the present embodiment is detected in the video flowing using optical flow method, frame difference method or background subtraction
Moving object；It is preferred that using background subtraction.The video flowing is detected using optical flow method, frame difference method or background subtraction
In moving object be in the prior art detect video flowing in moving object routine techniques, no longer it is carried out herein expansion retouch
It states.
Step S20: being directed to every frame motion picture, determine the region of moving object, in the background frame with it is described
The corresponding first area in region intercepts the first picture of the region.
Moving object region described in the present embodiment is that can accommodate the minimum rectangular area of the moving object.This
Embodiment can pass through 4 apex coordinates of region described in the every frame motion picture of calculating；It is looked in the background frame
To 4 the first coordinate points corresponding with 4 apex coordinates mode determine in the background frame with the region
Corresponding first area.The rectangular area that i.e. 4 first coordinate points are formed be in the background frame with the place
The corresponding first area in region.
Step S30: in real time by the first picture described in the first picture described in the background frame, every frame and every frame in institute
It states the corresponding first area in background frame and is uploaded to server end.
First picture described in the background frame and every frame is being uploaded to server end by collection terminal described in the present embodiment
Before, it also needs to encode the first picture described in the background frame and every frame, then executes above-mentioned steps S30 again.That is institute
It states after collection terminal encodes the first picture described in the background frame and every frame, in real time by the background painting after coding
First picture described in first picture described in every frame after face, coding and every frame is corresponding described in the background frame
One region is uploaded to server end.
First picture described in each frame described in the present embodiment carries corresponding frame motion picture in the video flowing
Timestamp.
Step S40: the server end is in real time by the received background frame of institute, first picture and every frame
It is handed down to client in first picture corresponding first area in the background frame.
Step S50: the client replicates according to the background frame for the first picture described in every frame and generates a frame the
One background frame affixes to corresponding first picture in first background frame in corresponding first area, forms a frame
All complete pictures are formed the first video flowing by complete picture.
Client described in the present embodiment receives the first picture described in the background frame and every frame that server end issues
Behind face, the first picture described in the background frame and every frame is decoded first, then executes the step S50 step again.
After the i.e. described client is to the first picture codec described in the background frame and every frame, drawn for described first after every frame decoding
Face generates first background frame of frame according to the decoded background frame duplication, the first picture after corresponding decoding is glued
It is labelled in first background frame in corresponding first area, forms a frame complete picture, by all complete pictures composition the
One video flowing.
Client described in the present embodiment by all complete pictures form the first video flowing specifically include: the client according to
According to the timestamp that corresponding first picture of every frame complete picture carries, all complete pictures are formed into first video flowing.
Step S60: rendering output is carried out to first video flowing.
The method of the processing of VR live video provided by above-described embodiment acquires background frame by collection terminal with after in real time
Continuous motion picture, the background frame and the motion picture form video flowing, detect the moving object in video flowing；For
Every frame motion picture determines the region of moving object, the firstth area corresponding with the region in the background frame
Domain intercepts the first picture of the region；In real time by the first picture described in the background frame, every frame and every frame institute
It states the first picture corresponding first area in the background frame and is uploaded to server end；The server end in real time will
First picture described in the received background frame of institute, first picture and every frame is corresponding in the background frame
It is handed down to client in the first area；The client is replicated for the first picture described in every frame according to the background frame
First background frame of frame is generated, corresponding first picture is affixed into corresponding first area in first background frame
In, a frame complete picture is formed, all complete pictures are formed into the first video flowing；Rendering output is carried out to first video flowing
Mode greatly reduce collection terminal to service under the premise of not influencing the clarity of client terminal playing VR live video
The data volume and reduce the data volume that server end is issued to client that device end uploads；To alleviate from collection terminal to
The uploading bandwidth pressure of server end, and alleviate and issue bandwidth pressure from server end to client.
The present invention further provides a kind of systems of VR live video processing.Referring to Fig. 2, Fig. 2 is that VR of the invention is broadcast live
The structural schematic diagram of one embodiment of system of video processing.In one embodiment, the system 100 of the VR live video processing is wrapped
It includes: collection terminal 110, server end 120, client 130.Wherein, the collection terminal 110, in real time acquire background frame and
Subsequent motion picture, the background frame and the motion picture form video flowing, detect the moving object in video flowing；Needle
To every frame motion picture, the region of moving object is determined, corresponding with the region first in the background frame
Region intercepts the first picture of the region；And in real time by the first picture described in the background frame, every frame and
The corresponding first area in the background frame of first picture described in every frame is uploaded to server end 120.The service
Device end 120, in real time by the first picture described in the received background frame, first picture and every frame in institute
It states the corresponding first area in background frame and is handed down to client 130.The client 130, for for described in every frame
First picture replicates according to the background frame and generates first background frame of frame, corresponding first picture is affixed to described
In first background frame in corresponding first area, a frame complete picture is formed, all complete pictures are formed into the first video flowing；
Rendering output is carried out to first video flowing.
Collection terminal 110 described in the present embodiment detects the video flowing using optical flow method, frame difference method or background subtraction
In moving object；It is preferred that using background subtraction.The video is detected using optical flow method, frame difference method or background subtraction
Moving object in stream is the routine techniques for detecting moving object in video flowing in the prior art, and no longer it is unfolded herein
Description.
Moving object region described in the present embodiment is that can accommodate the minimum rectangular area of the moving object.This
Embodiment can pass through 4 apex coordinates of region described in the every frame motion picture of calculating；It is looked in the background frame
To 4 the first coordinate points corresponding with 4 apex coordinates mode determine in the background frame with the region
Corresponding first area.The rectangular area that i.e. 4 first coordinate points are formed be in the background frame with the place
The corresponding first area in region.
First picture described in the background frame and every frame is being uploaded to server by collection terminal 110 described in the present embodiment
Before end 120, also needs to encode the first picture described in the background frame and every frame, then execute the background painting again
First picture described in first picture described in face, every frame and every frame is in the background frame on the corresponding first area
Reach the operation of server end.After the i.e. described collection terminal 110 encodes the first picture described in the background frame and every frame,
In real time by the first picture described in the first picture described in the background frame after coding, every frame after coding and every frame in institute
It states the corresponding first area in background frame and is uploaded to server end 120.
Client 130 described in the present embodiment receives described in the background frame and every frame that server end 120 issues
After first picture, the first picture described in the background frame and every frame is decoded first, then executing again described will be connect
First picture described in the background frame, first picture and the every frame received is corresponding described in the background frame
It is handed down to the operation of client 130 in first area.The i.e. described client 130 is to the first picture described in the background frame and every frame
After decoding, for first picture after every frame decoding, a frame first is generated according to the decoded background frame duplication
Background frame affixes to the first picture after corresponding decoding in first background frame in corresponding first area, is formed
All complete pictures are formed the first video flowing by one frame complete picture.
Each the first picture of frame described in the present embodiment carries time of the corresponding frame motion picture in the video flowing
Stamp.The client 130 will be all specifically for the timestamp carried according to corresponding first picture of every frame complete picture
Complete picture forms first video flowing.
The system 100 of the processing of VR live video provided by above-described embodiment acquires background painting by collection terminal 110 in real time
Face and subsequent motion picture, the background frame and the motion picture form video flowing, detect the moving object in video flowing
Body；For every frame motion picture, the region of moving object, corresponding with the region in the background frame is determined
First area intercepts the first picture of the region；In real time by the first picture described in the background frame, every frame and
The corresponding first area in the background frame of first picture described in every frame is uploaded to server end 120；The service
Device end 120 is in real time by the first picture described in the received background frame of institute, first picture and every frame in the background
Client 130 is handed down in the corresponding first area in picture；The client 130 is for the first picture described in every frame, root
It is replicated according to the background frame and generates first background frame of frame, corresponding first picture is affixed into first background frame
In in corresponding first area, form a frame complete picture, all complete pictures formed into the first video flowing；To first view
The mode that frequency stream carries out rendering output greatly subtracts under the premise of not influencing the clarity of client terminal playing VR live video
The data volume and reduce what server end 120 was issued to client 130 that collection terminal 110 is uploaded to server end 120 are lacked
Data volume；To alleviate the uploading bandwidth pressure from collection terminal 110 to server end 120, and alleviate from server end
120 issue bandwidth pressure to client 130.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (6)
1. a kind of method of VR live video processing, which is characterized in that the method for the VR live video processing includes:
Collection terminal acquires background frame and subsequent motion picture in real time, and the background frame and the motion picture form video
Stream detects the moving object in video flowing；
For every frame motion picture, the region of moving object, corresponding with the region in the background frame is determined
First area, intercept the first picture of the region；
In real time by the first picture described in the first picture described in the background frame, every frame and every frame in the background frame
The corresponding first area is uploaded to server end；
The server end is in real time by the first picture described in the received background frame of institute, first picture and every frame
Client is handed down in the corresponding first area in the background frame；
The client replicates according to the background frame for the first picture described in every frame and generates first background frame of frame,
Corresponding first picture is affixed in first background frame in corresponding first area, a frame complete picture is formed, it will
All complete pictures form the first video flowing；
Rendering output is carried out to first video flowing.
2. the method for VR live video processing according to claim 1, which is characterized in that the collection terminal uses light stream
Method, frame difference method or background subtraction detect the moving object in the video flowing.
3. the method for VR live video processing according to claim 1, which is characterized in that each first picture of frame is taken
Timestamp with corresponding frame motion picture in the video flowing；
All complete pictures are formed the first video flowing and specifically included by the client:
The timestamp that the client is carried according to corresponding first picture of every frame complete picture, all complete pictures are formed
First video flowing.
4. a kind of system of VR live video processing, which is characterized in that the system of the VR live video processing includes:
Collection terminal, for acquiring background frame and subsequent motion picture, the background frame and the motion picture shape in real time
At video flowing, the moving object in video flowing is detected；For every frame motion picture, the region, described of moving object is determined
First area corresponding with the region in background frame, intercepts the first picture of the region；And in real time will
First picture described in first picture described in the background frame, every frame and every frame is corresponding described in the background frame
First area is uploaded to server end；
The server end, in real time will be first described in the received background frame of institute, first picture and every frame
It is handed down to client in picture corresponding first area in the background frame；
The client, for being replicated according to the background frame and generating first background of frame for the first picture described in every frame
Picture affixes to corresponding first picture in first background frame in corresponding first area, forms a frame and completely draws
All complete pictures are formed the first video flowing by face；Rendering output is carried out to first video flowing.
5. the system of VR live video processing according to claim 4, which is characterized in that the collection terminal uses light stream
Method, frame difference method or background subtraction detect the moving object in the video flowing.
6. the system of VR live video processing according to claim 4, which is characterized in that each first picture of frame is taken
Timestamp with corresponding frame motion picture in the video flowing；
The client will be all complete specifically for the timestamp carried according to corresponding first picture of every frame complete picture
Whole picture forms first video flowing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811366415.5A CN109729368A (en) | 2018-11-16 | 2018-11-16 | The method and system of VR live video processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811366415.5A CN109729368A (en) | 2018-11-16 | 2018-11-16 | The method and system of VR live video processing |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109729368A true CN109729368A (en) | 2019-05-07 |
Family
ID=66295110
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811366415.5A Pending CN109729368A (en) | 2018-11-16 | 2018-11-16 | The method and system of VR live video processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109729368A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112702611A (en) * | 2019-10-22 | 2021-04-23 | 上海华为技术有限公司 | Playing method and playing system |
-
2018
- 2018-11-16 CN CN201811366415.5A patent/CN109729368A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112702611A (en) * | 2019-10-22 | 2021-04-23 | 上海华为技术有限公司 | Playing method and playing system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jansen et al. | A pipeline for multiparty volumetric video conferencing: transmission of point clouds over low latency DASH | |
| WO2017181777A1 (en) | Panoramic live video streaming method, device, system, and video source control apparatus | |
| CN108076349B (en) | Network interactive live broadcast method, system and electronic equipment | |
| US20080168512A1 (en) | System and Method to Implement Interactive Video Streaming | |
| CN103905741A (en) | Ultra-high-definition panoramic video real-time generation and multi-channel synchronous play system | |
| US11282169B2 (en) | Method and apparatus for processing and distributing live virtual reality content | |
| CN106604042A (en) | Panorama webcasting system and panorama webcasting method based on cloud server | |
| CN110809173B (en) | Virtual live broadcast method and system based on AR augmented reality of smart phone | |
| CN105847718A (en) | Scene recognition-based live video bullet screen display method and display device thereof | |
| US11120615B2 (en) | Dynamic rendering of low frequency objects in a virtual reality system | |
| Schreer et al. | Lessons learned during one year of commercial volumetric video production | |
| CN105933707B (en) | A kind of compression method and device of panoramic video | |
| CN109729368A (en) | The method and system of VR live video processing | |
| CN110433491A (en) | Movement sync response method, system, device and the storage medium of virtual spectators | |
| US12112426B2 (en) | Augmenting a view of a real-world environment with a view of a volumetric video object | |
| CN115988171B (en) | Video conference system and immersive layout method and device thereof | |
| CN112019921A (en) | Body motion data processing method applied to virtual studio | |
| CN109600623A (en) | The control method and system of VR live video | |
| CN111447458A (en) | Live broadcast system, method and device based on content explanation and live broadcast server | |
| CN106060518A (en) | Method and system for implementing 720-degree panoramic player with view angle switching function | |
| KR102265784B1 (en) | Apparatus and method for client side forensic watermark | |
| CN113630575B (en) | Method, system and storage medium for displaying images of multi-person online video conference | |
| KR102404130B1 (en) | Device for transmitting tele-presence image, device for receiving tele-presence image and system for providing tele-presence image | |
| CN116962744A (en) | Live webcast link interaction method, device and live broadcast system | |
| CN113395527A (en) | Remote live broadcast virtual background cloud synthesis system based on VR technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190507 |
|
| WD01 | Invention patent application deemed withdrawn after publication |