CN106331823B - A kind of video broadcasting method and device - Google Patents
A kind of video broadcasting method and device Download PDFInfo
- Publication number
- CN106331823B CN106331823B CN201610799113.1A CN201610799113A CN106331823B CN 106331823 B CN106331823 B CN 106331823B CN 201610799113 A CN201610799113 A CN 201610799113A CN 106331823 B CN106331823 B CN 106331823B
- Authority
- CN
- China
- Prior art keywords
- buffering
- image
- panoramic picture
- panoramic
- motion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000003139 buffering effect Effects 0.000 claims abstract description 144
- 239000000872 buffer Substances 0.000 claims abstract description 54
- 239000000284 extract Substances 0.000 claims abstract description 15
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 9
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 8
- 230000008676 import Effects 0.000 claims description 10
- 230000004886 head movement Effects 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 238000002156 mixing Methods 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims description 5
- 238000004080 punching Methods 0.000 claims description 4
- 239000002131 composite material Substances 0.000 claims description 2
- 238000001514 detection method Methods 0.000 claims description 2
- 238000007726 management method Methods 0.000 description 17
- 238000010586 diagram Methods 0.000 description 16
- 238000012545 processing Methods 0.000 description 7
- 238000004590 computer program Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000011049 filling Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000002194 synthesizing effect Effects 0.000 description 4
- 241000282693 Cercopithecidae Species 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 238000007654 immersion Methods 0.000 description 3
- 208000012886 Vertigo Diseases 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 208000002173 dizziness Diseases 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000004904 shortening Methods 0.000 description 2
- 238000003860 storage Methods 0.000 description 2
- 230000026676 system process Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4305—Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
- H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44016—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for substituting a video clip
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Processing Or Creating Images (AREA)
- Studio Circuits (AREA)
Abstract
The embodiment of the invention provides a kind of video broadcasting method and devices, wherein video broadcasting method includes that panoramic picture buffering is formed according to panoramic video data, and by panoramic picture buffer fillings into buffering queue, panoramic picture buffering is obtained from buffering queue again and carries out image management, to obtain pretreatment image buffering.To the motion-sensing data acquired according to motion sensor, pretreatment image buffering extract by system and generates scene image buffering, to enable a system to according to newest sensing data, synthesis can be carried out after extracting panoramic picture buffering to show, picture delay time caused by substantially shorten because of user movement, and then promote user's use feeling.
Description
Technical field
The present embodiments relate to the communications field more particularly to a kind of video broadcasting methods and device.
Background technique
With the fast development of VR (Virtual Reality, virtual reality) technology in recent years, VR product user is not yet
It is disconnected to increase.For VR product user, most strong use feeling is the feeling of immersion to broadcasting content.
In the prior art, user loses feeling of immersion, seriously affects using generally existing spinning sensation in VR product process
Use feeling.It is mainly too long caused by picture delay time to generate this spinning sensation, wherein picture delay refers to user
(head) movement starts, and sees that post exercise picture is fully displayed at the time interval on terminal device to user.When between the time
When more than 20ms, user will perceive picture delay.The too long meeting of picture delay time is so that human body sensing technical ability and human eye
The image seen mismatches, and then generates the feeling of dizziness.
Therefore, the picture delay for how reducing panoramic video broadcasting, is a technical problem to be solved urgently.
Summary of the invention
The embodiment of the present invention provides a kind of video broadcasting method, and head turns when solving panoramic video broadcasting in the prior art
Picture caused by dynamic postpones excessive problem.
To solve the above-mentioned problems, the invention discloses a kind of video broadcasting methods, comprising:
Panoramic picture buffering is formed according to panoramic video data, and by the panoramic picture buffer fillings to buffering queue
In, the panoramic picture buffering includes that panorama marks.
The panoramic picture buffering is obtained from the buffering queue carries out image management, it is slow to obtain pretreatment image
Punching.
According to the motion-sensing data that motion sensor acquires, pretreatment image buffering is extracted by system
Generate scene image buffering.
Scene image buffering is synthesized, to show the scene image of corresponding position.
Preferably, panoramic picture buffering is formed according to panoramic video data, and by the panoramic picture buffer fillings to slow
It rushes in queue, including removes free buffer from buffering queue；The panoramic video data that application program obtains are filled to described
In free buffer, panoramic picture buffering is formed；Panoramic picture buffering is moved into the buffer queue.
Preferably, pretreatment image buffering extract before generating scene image buffering by system, further includes: inspection
It surveys the pretreatment image buffering to mark with the presence or absence of panorama, is that panoramic picture buffers with the determination pretreatment image buffering.
Preferably, pretreatment image buffering extract by system and generate scene image buffering, comprising: pass through system
Blending algorithm is called to extract panoramic picture buffering, to generate the scene image buffering.
Wherein, pretreatment image buffering is extracted, is synthesized with to scene image buffering, in same frame synchronization
It completes, and shows the scene image of corresponding position in next frame synchronization.
Further, motion sensor is set in virtual reality device, for being tracked to user's head movement, is obtained
Fetching shows the motion-sensing data of head movement state.
On the other hand, to solve the above-mentioned problems, the invention also discloses a kind of video play devices, comprising:
Image import modul for forming panoramic picture buffering according to panoramic video data, and the panoramic picture is delayed
Filling is charged in buffering queue, and the panoramic picture buffering includes that panorama marks.
Image management module carries out image management for obtaining the panoramic picture buffering from the buffering queue, with
Obtain pretreatment image buffering.
Abstraction module, the motion-sensing data for being acquired according to motion sensor scheme the pretreatment by system
Scene image buffering is generated as buffering extract.
Display module is synthesized, for synthesizing scene image buffering, to show the scene image of corresponding position.
To sum up, the embodiment of the present invention, which passes through, forms panoramic picture buffering according to panoramic video data, and panoramic picture is buffered
Filling is into buffering queue.And obtain panoramic picture buffering from buffering queue and carry out image management, to obtain pretreatment figure
As buffering.Further according to the motion-sensing data of motion sensor acquisition, extraction life is carried out to pretreatment image buffering by system
It is buffered at scene image, it, can after being extracted to panoramic picture buffering to enable a system to according to newest sensing data
Synthesis is carried out to show, thus picture delay time caused by substantially shortening because of user movement, and then make user that can not perceive picture
Face delay, promotes user's use feeling.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below by institute in the description to the embodiment of the present invention
Attached drawing to be used is needed to be briefly described, it should be apparent that, the accompanying drawings in the following description is only some implementations of the invention
Example, for those of ordinary skill in the art, without any creative labor, can also be according to these attached drawings
Obtain other attached drawings.
Fig. 1 is a kind of video broadcasting method flow chart provided in an embodiment of the present invention；
Fig. 2 is another video broadcasting method flow chart provided in an embodiment of the present invention；
Fig. 3 is that a kind of buffer of video broadcasting method in the prior art manages schematic diagram；
Fig. 4 is a kind of time diagram of video broadcasting method in the prior art；
Fig. 5 is that a kind of buffer of video broadcasting method provided in an embodiment of the present invention manages schematic diagram；
Fig. 6 is a kind of time diagram of video broadcasting method provided in an embodiment of the present invention；
Fig. 7 is a kind of structural block diagram of video play device provided in an embodiment of the present invention；
Fig. 8 is the structural block diagram of another video play device provided in an embodiment of the present invention.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair
Embodiment in bright, every other implementation obtained by those of ordinary skill in the art without making creative efforts
Example, shall fall within the protection scope of the present invention.
Provided video broadcasting method to facilitate the understanding of the present invention, before describing the specific embodiments, to embodiment
Involved in technical term explain:
GraphicBuffer: image buffers, android system are described by defining GraphicBuffer data type
One piece of image buffer, the object can be transmitted with striding course.
Bufferqueue: buffering queue is that the local of ISurfaceTexture is realized, it is necessary to each void in reloading interface
Function, such as queueBuffer, dequeueBuffer etc..It is considered that BufferQueue is a service centre, pass through it
To manage buffer.
SurfaceFlinger: image management module is mainly responsible for the Surface for synthesizing each window, then passes through
OpenGLES is shown on FrameBuffer.System process of the SurfaceFlinger service operation in android system
In, it is responsible for managing the frame buffer zone of android system.
Hwcomposer: image synthesizer completes the function that image data set merges display using hardware.
Surface: drawing object has corresponded to one piece of screen buffer, and each window corresponds to a Surface, any
View is drawn on Surface.
Queuebuffer: buffering is moved into queue, can also be expressed as arriving buffer queue by buffering of falling in lines
In Bufferqueue.
Dequeuebuffer: buffering of falling out removes buffering that is, from queue, can also be expressed as from Bufferqueue
Dequeue goes out buffer.
VR (Virtual Reality, virtual reality): being a kind of Computer Simulation that can be created with the experiencing virtual world
System.It generates a kind of simulated environment using computer, be a kind of Multi-source Information Fusion interactive Three-Dimensional Dynamic what comes into a driver's and
The system emulation of entity behavior is able to use family and is immersed in the environment.
FrameBuffer: frame buffering, for passing through picture output device from the core buffer comprising complete frame data
In drive video display apparatus.The color-values composition of each pixel on screen is contained on core buffer Plays.
Overlay: superimposed layer in the case where not modifying framework source code, realizes the customization of resource.
The brief explanation meaning of technical term herein, the above technical term can it is subsequent in conjunction with specific embodiments carry out into
Explain to one step.
Embodiment one
Referring to Fig.1, a kind of video broadcasting method flow chart provided in an embodiment of the present invention is given.
Step 101, panoramic picture buffering is formed according to panoramic video data, and by panoramic picture buffer fillings to buffering team
In column.
Wherein, panoramic video is obtained by application program, including 360 degree of panoramic videos, is whole by capturing to mm professional camera special
The image information of a scene or the picture after use modeling software rendering, carry out picture split using software, and with specially
The panoramic picture that plays out of player.It is browsed for virtual reality, two-dimensional plan view is modeled to true three-dimensional space
Between, it is presented to the user.
In the present embodiment, for being applied to the field VR.It, can be in panorama in order to determine that the video of broadcasting is panoramic video
Panorama is set in the image buffers of video and marks flag, then correspondence image is panoramic video to flag if it exists.
By filling panoramic video data into the free buffer buffer1 of buffering queue Bufferqueue, formed complete
Scape image buffers GraphicBuffer1 is directed into the panoramic video data in system cache, convenient for system to the aphorama
Frequency is operated.
Step 102, panoramic picture buffering is obtained from buffering queue carries out image management, it is slow to obtain pretreatment image
Punching.
Wherein, image management module SurfaceFlinger obtains the GraphicBuffer1 inside Bufferqueue,
And image management is carried out, it obtains pretreatment image and buffers GraphicBuffer2.Wherein, image management is specifically used for each window
Mouth is measured, is laid out and drawn, to be adjusted to buffering.
Step 103, the motion-sensing data acquired according to motion sensor buffer pretreatment image by system and carry out
It extracts and generates scene image buffering.
Wherein, motion sensor is for detecting user when watching the panoramic video, the motion conditions on head.For example, working as
When head rotation, motion sensor positioning head is rotated after visual direction, and using the information as motion-sensing data send out
It send into system.So that system extracts according to the motion-sensing data scene image in visual direction from GraphicBuffer2
Buffer GraphicBuffer3.Step 104, scene image buffering is synthesized, to show the scene image of corresponding position.
GraphicBuffer3 is synthesized using image synthesizer Hwcomposer, thus in virtual reality device
The scene image in visual direction after user's head moves is shown on screen.
Due to, by the data import system caching of panoramic video, being converted panoramic video data in abovementioned steps
GraphicBuffer2.It therefore, only need to be to GraphicBuffer2 after motion sensor collects this motion-sensing data
It is extracted, GraphicBuffer3 can be directly placed into Hwcomposer and be synthesized, and be displayed on the screen.Without
Panoramic video data are extracted by application program, then result import system will be extracted, and by system to image buffers
After processing, it can just be put into Hwcomposer and synthesize, and be displayed on the screen.I.e. the application obtains motion-sensing number to reduce
According to operation later to shorten the time, all video image data are carried out buffered in advance.When motion sensor obtains
When to the position of head rotation, it is only necessary to be selected from the GraphicBuffer2 for be completed buffered and motion-sensing number
According to corresponding GraphicBuffer3.It is shown to display screen to the end to be greatly shortened from acquisition motion-sensing data
The upper required time.
To sum up, the embodiment of the present invention, which passes through, forms panoramic picture buffering according to panoramic video data, and panoramic picture is buffered
Filling is into buffering queue.And obtain panoramic picture buffering from buffering queue and carry out image management, to obtain pretreatment figure
As buffering.Further according to the motion-sensing data of motion sensor acquisition, extraction life is carried out to pretreatment image buffering by system
It is buffered at scene image, it, can after being extracted to panoramic picture buffering to enable a system to according to newest sensing data
Synthesis is carried out to show, thus picture delay time caused by substantially shortening because of user movement, and then make user that can not perceive picture
Face delay, promotes user's use feeling.
Embodiment two
On the basis of the above embodiments, the present embodiment is discussed further video broadcasting method.
Referring to Fig. 2, another video broadcasting method flow chart provided in an embodiment of the present invention is given:
Step 201, panoramic picture buffering is formed according to panoramic video data, and by panoramic picture buffer fillings to buffering team
In column.
Specifically, application program dequeue from BufferQueue goes out buffer1 with carry image information, journey will be applied
After the panoramic video data that sequence obtains are filled into the buffer1 to form GraphicBuffer1, then should
In GraphicBuffer1queue to BufferQueue, to complete importing of the panoramic video data to system.
Step 202, panoramic picture buffering is obtained from buffering queue carries out image management.
Specifically, after panoramic video data are filled into BufferQueue, by SurfaceFlinger from
The GraphicBuffer1 is obtained in BufferQueue, and image management is carried out to the GraphicBuffer1, is obtained
GraphicBuffer2.After GraphicBuffer2 is called by subsequent step, it will buffer1 discharge to
In BufferQueue, wherein SurfaceFlinger service operation is specifically used for management in the System process of Android system
The frame of Android system buffers FrameBuffer, synthesizes the Surface of each window.
Step 203, the pretreatment image buffering is detected to mark with the presence or absence of panorama.
Specifically, since the image buffers that system is panoramic video are provided with flag, it can be right
GraphicBuffer2 extract before generating GraphicBuffer3, judges the image there are flag by detecting whether
Whether buffering includes panoramic video.Flag if it exists then needs to extract the GraphicBuffer2.
Step 204, the motion-sensing data acquired according to motion sensor buffer pretreatment image by system and carry out
It extracts and generates scene image buffering.
In the present embodiment, for being applied to the field VR.Motion sensor can be set in VR equipment, for example, can be with
It being set in the VR helmet, wherein the VR helmet is a kind of using head-mounted display, by being closed to extraneous vision, the sense of hearing for people,
Guidance user generates a kind of virtual reality terminal of feeling in virtual environment.When user's head movement when, can to
Family head movement is tracked, to obtain the motion-sensing data of instruction head movement state.
When motion sensor collects motion-sensing data, due to existing corresponding scene in caching
GraphicBuffer2, therefore can be according to newest motion-sensing data, directly by system to panoramic video
GraphicBuffer2 is extracted, to obtain GraphicBuffer3 corresponding with motion-sensing data.Wherein, to aphorama
The GraphicBuffer2 of frequency is extracted, and blending algorithm can be called to complete by system.
Step 205, scene image buffering is synthesized, to show the scene image of corresponding position.
Specifically, being synthesized GraphicBuffer3 by Hwcomposer, obtain corresponding with motion-sensing data
The scene image of position, and the scene image is shown on the display screen of virtual display device.Wherein, if passing through overlay
GraphicBuffer2 is then directly synthesized on overlay by Hwcomposer by mode.If by framebuffer mode,
Then the GraphicBuffer2 is plotted on framebuffer, and is synthesized to framebuffer eventually by Hwcomposer
On.
Unlike the prior art, when watching panoramic video using virtual display device, if head is turned to by position A
Position B.In the prior art, it after motion sensor collects this motion-sensing data, needs through application program to complete
Scape video data is extracted, then will extract result import system, and by system to image buffers processing after, can just be put into
It synthesizes, and is displayed on the screen in Hwcomposer.It can be seen that from this process from acquisition motion-sensing data and show to the end
Show on display screen, at least needs the time of 3-4 frame synchronization.A kind of video broadcasting method in the prior art is given referring to Fig. 3
Buffer manage schematic diagram.A kind of time diagram of video broadcasting method in the prior art is given referring to Fig. 4.Even if adopting
With " three bufferings " technology, the time of picture delay is also required to the time more than 2 frame synchronization.If the display screen of virtual reality device
Use the refresh rate of current mainstream for the liquid crystal display of 60Hz, then the display screen shows that the time of a frame is about 16.67ms, shows
Showing that 2 frames i.e. 2 frame synchronization just need is more than the time of 33ms, i.e. picture delay time will make user clearly far more than 20ms
Feel picture postpone.For example, you see a monkey in panoramic video, when monkey from your visual field rapidly,
When ghost is climbed on by the side of one tree in tow, you look up the monkey on tree, have spent the time more than 20ms, picture just with
Your head turned in the past.So that user loses feeling of immersion.
And the embodiment of the present invention is due to by the data import system of panoramic video caching, and generates
GraphicBuffer2.So after motion sensor collects this motion-sensing data, it only need to be to GraphicBuffer2
It is extracted, can will extract obtained GraphicBuffer3 and be directly placed into Hwcomposer and synthesize, and be shown in screen
On curtain.The time of picture delay is greatly shortened.A kind of video broadcasting method provided in an embodiment of the present invention is given referring to Fig. 5
Buffer manage schematic diagram.A kind of timing signal of video broadcasting method provided in an embodiment of the present invention is given referring to Fig. 6
Figure.Wherein, GraphicBuffer2 is extracted, is synthesized with to GraphicBuffer3, it can be in same frame synchronization
Middle completion, therefore in next frame synchronization that motion sensor collects this motion-sensing data, it can show correspondence
The scene image of position, so that the time of picture delay to be controlled to the time in a frame synchronization.For example, if virtual reality device
Display screen uses the refresh rate of current mainstream for the liquid crystal display of 60Hz, then the display screen shows that the time of a frame is about
16.67ms, i.e. picture delay time can control within 20ms, and then make user that can not perceive picture delay, avoid
Because the raw dizziness of picture delay artificial delivery is felt, user's use feeling is promoted.
It should be noted that for simple description, therefore, it is stated as a series of action groups for embodiment of the method
It closes, but those skilled in the art should understand that, embodiment of that present invention are not limited by the describe sequence of actions, because according to
According to the embodiment of the present invention, some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art also should
Know, the embodiments described in the specification are all preferred embodiments, and the related movement not necessarily present invention is implemented
Necessary to example.
Embodiment three
On the basis of the above embodiments, the present embodiment additionally provides a kind of video play device, is applied to virtual reality
Terminal.
A kind of structural block diagram of video play device provided in an embodiment of the present invention is given referring to Fig. 7, specifically can wrap
Include following module:
Image import modul 701, for forming panoramic picture buffering according to panoramic video data, and by the panoramic picture
For buffer fillings into buffering queue, panoramic picture buffering includes that panorama marks.
Image management module 702 carries out image management for obtaining panoramic picture buffering from buffering queue, pre- to obtain
Handle image buffers.
Abstraction module 703, the motion-sensing data for being acquired according to motion sensor, by system to pretreatment image
Buffering, which extract, generates scene image buffering.
Display module 704 is synthesized, for synthesizing scene image buffering, to show the scene image of corresponding position.
Referring to Fig. 8, in an optional implementation of the invention, on the basis of Fig. 7, video play device is also wrapped
It includes:
Panorama detection module 705, for examining before carrying out extracting generation scene image buffering to pretreatment image buffering
It surveys pretreatment image buffering to mark with the presence or absence of panorama, to determine that pretreatment image buffering is that panoramic picture buffers.
Image import modul 701 includes: fall out submodule 7011, filling submodule 7012 and submodule 7013 of falling in lines.Its
In, submodule 7011 of falling out, for removing free buffer from buffering queue；Submodule 7012 is filled, is used for application program
The panoramic video data of acquisition are filled into free buffer, and panoramic picture buffering is formed；Fall in lines submodule 7013, for by panorama
Image buffers move into buffer queue.
Based on this, abstraction module 703 is taken out specifically for calling blending algorithm to buffer panoramic picture by system
It takes, to generate scene image buffering.
Further, abstraction module 703 extracts pretreatment image buffering, with synthesis display module 704 to scene
Image buffers are synthesized, and are completed in same frame synchronization；And display of the display module 704 to composite result is synthesized, next
It is completed in frame synchronization.
Wherein, motion sensor is set in virtual reality device, and for being tracked to user's head movement, acquisition refers to
Show the motion-sensing data of head movement state.
To sum up, the embodiment of the present invention forms panoramic picture buffering according to panoramic video data by image import modul 701,
By panoramic picture buffer fillings into buffering queue.And the panorama sketch is obtained from buffering queue by image management module 702
As buffering progress image management, to obtain pretreatment image buffering.Further according to the motion-sensing data of motion sensor acquisition, lead to
Abstraction module 703 is crossed pretreatment image buffering is carried out to extract generation scene image buffering, it is aobvious by synthesizing to enable a system to
Show module 704 according to newest sensing data, can the extraction result to abstraction module 703 carry out synthesis and show, thus greatly
Picture delay time caused by width shortens because of user movement, and then make user that can not perceive picture delay, it promotes user and uses
Impression.
For device embodiment, since it is basically similar to the method embodiment, related so being described relatively simple
Place illustrates referring to the part of embodiment of the method.
All the embodiments in this specification are described in a progressive manner, the highlights of each of the examples are with
The difference of other embodiments, the same or similar parts between the embodiments can be referred to each other.
It should be understood by those skilled in the art that, the embodiment of the embodiment of the present invention can provide as method, apparatus or calculate
Machine program product.Therefore, the embodiment of the present invention can be used complete hardware embodiment, complete software embodiment or combine software and
The form of the embodiment of hardware aspect.Moreover, the embodiment of the present invention can be used one or more wherein include computer can
With in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code
The form of the computer program product of implementation.
The embodiment of the present invention be referring to according to the method for the embodiment of the present invention, terminal device (system) and computer program
The flowchart and/or the block diagram of product describes.It should be understood that flowchart and/or the block diagram can be realized by computer program instructions
In each flow and/or block and flowchart and/or the block diagram in process and/or box combination.It can provide these
Computer program instructions are set to general purpose computer, special purpose computer, Embedded Processor or other programmable data processing terminals
Standby processor is to generate a machine, so that being held by the processor of computer or other programmable data processing terminal devices
Capable instruction generates for realizing in one or more flows of the flowchart and/or one or more blocks of the block diagram
The device of specified function.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing terminal devices
In computer-readable memory operate in a specific manner, so that instruction stored in the computer readable memory generates packet
The manufacture of command device is included, which realizes in one side of one or more flows of the flowchart and/or block diagram
The function of being specified in frame or multiple boxes.
These computer program instructions can also be loaded into computer or other programmable data processing terminal devices, so that
Series of operation steps are executed on computer or other programmable terminal equipments to generate computer implemented processing, thus
The instruction executed on computer or other programmable terminal equipments is provided for realizing in one or more flows of the flowchart
And/or in one or more blocks of the block diagram specify function the step of.
Although the preferred embodiment of the embodiment of the present invention has been described, once a person skilled in the art knows bases
This creative concept, then additional changes and modifications can be made to these embodiments.So the following claims are intended to be interpreted as
Including preferred embodiment and fall into all change and modification of range of embodiment of the invention.
Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by
One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation
Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning
Covering non-exclusive inclusion, so that process, method, article or terminal device including a series of elements not only wrap
Those elements are included, but also including other elements that are not explicitly listed, or further includes for this process, method, article
Or the element that terminal device is intrinsic.In the absence of more restrictions, being wanted by what sentence "including a ..." limited
Element, it is not excluded that there is also other identical elements in process, method, article or the terminal device for including element.
Above to a kind of video broadcasting method provided by the present invention and device, it is described in detail, it is used herein
A specific example illustrates the principle and implementation of the invention, and the above embodiments are only used to help understand
Method and its core concept of the invention；At the same time, for those skilled in the art is having according to the thought of the present invention
There will be changes in body embodiment and application range, and to sum up, the content of the present specification should not be construed as to limit of the invention
System.
Claims (10)
1. a kind of video broadcasting method, which is characterized in that the described method includes:
Panoramic picture buffering is formed according to panoramic video data, and by the panoramic picture buffer fillings into buffering queue, institute
Stating panoramic picture buffering includes that panorama marks；Wherein, the panoramic picture buffering is complete for describing one piece in Android system
The data type of scape image buffers；
The panoramic picture buffering is obtained from the buffering queue and carries out image management, to obtain pretreatment image buffering；Its
In, described image management is for measuring, being laid out and drawing to each window, to adjust buffering；
According to the motion-sensing data that motion sensor acquires, visually side is extracted from pretreatment image buffering by system
To scene image buffering；
Scene image buffering is synthesized, to show the scene image of corresponding position；
Wherein, described that pretreatment image buffering is extracted, scene image buffering is synthesized with described, it is same in same frame
It is completed in step, the scene image of the corresponding position is shown in next frame synchronization.
2. the method according to claim 1, wherein described slow according to panoramic video data formation panoramic picture
Punching, and by the panoramic picture buffer fillings into buffering queue, comprising:
Free buffer is removed from buffering queue；
The panoramic video data that application program obtains are filled into the free buffer, panoramic picture buffering is formed；
Panoramic picture buffering is moved into the buffer queue.
3. the method according to claim 1, wherein described extract pretreatment image buffering by system
Before generation scene image buffering, further includes:
It detects the pretreatment image buffering to mark with the presence or absence of panorama, with the determination pretreatment image buffering for panoramic picture
Buffering.
4. the method according to claim 1, wherein described extract pretreatment image buffering by system
Generate scene image buffering, comprising:
Blending algorithm is called to extract panoramic picture buffering by system, to generate the scene image buffering.
5. the method according to claim 1, wherein
The motion sensor is set in virtual reality device, for being tracked to user's head movement, obtains instruction head
The motion-sensing data of portion's motion state.
6. a kind of video play device, which is characterized in that described device includes:
Image import modul for forming panoramic picture buffering according to panoramic video data, and panoramic picture buffering is filled out
It is charged in buffering queue, the panoramic picture buffering includes that panorama marks；Wherein, the panoramic picture buffering is in Android
The data type of one block of panoramic picture buffering is described in system；
Image management module carries out image management for obtaining the panoramic picture buffering from the buffering queue, to obtain
Pretreatment image buffering；Wherein, described image management is for measuring, being laid out and drawing to each window, to adjust buffering；
Abstraction module, the motion-sensing data for being acquired according to motion sensor are slow from the pretreatment image by system
The scene image buffering in visual direction is extracted in punching；
Display module is synthesized, for closing system according to the scene image of motion-sensing data pick-up buffering
At to show the scene image of corresponding position；
Wherein, the abstraction module extracts pretreatment image buffering, slow to scene image with the synthesis display module
Capable synthesis is rushed in, is completed in same frame synchronization, display of the synthesis display module to composite result, in next frame synchronization
It completes.
7. device according to claim 6, which is characterized in that described image import modul includes:
It falls out submodule, for removing free buffer from buffering queue；
Submodule is filled, the panoramic video data for obtaining application program are filled into the free buffer, and panorama is formed
Image buffers；
Fall in lines submodule, for panoramic picture buffering to be moved into the buffer queue.
8. device according to claim 6, which is characterized in that described device further include:
Panorama detection module is marked for detecting the pretreatment image buffering with the presence or absence of panorama, with the determination pretreatment
Image buffers are panoramic picture buffering.
9. device according to claim 6, which is characterized in that
The abstraction module, specifically for calling blending algorithm to extract panoramic picture buffering by system, to generate
State scene image buffering.
10. device according to claim 6, which is characterized in that
The motion sensor is set in virtual reality device, for being tracked to user's head movement, obtains instruction head
The motion-sensing data of portion's motion state.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610799113.1A CN106331823B (en) | 2016-08-31 | 2016-08-31 | A kind of video broadcasting method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610799113.1A CN106331823B (en) | 2016-08-31 | 2016-08-31 | A kind of video broadcasting method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106331823A CN106331823A (en) | 2017-01-11 |
| CN106331823B true CN106331823B (en) | 2019-08-20 |
Family
ID=57786299
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610799113.1A Active CN106331823B (en) | 2016-08-31 | 2016-08-31 | A kind of video broadcasting method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106331823B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106896826B (en) * | 2017-02-27 | 2020-04-28 | 张斌 | Motion scene digital synthesis system and method |
| CN109791297B (en) * | 2017-07-18 | 2021-07-20 | 深圳市柔宇科技股份有限公司 | Method for detecting data transmission delay of head-mounted display and head-mounted display |
| CN107743223A (en) * | 2017-10-31 | 2018-02-27 | 歌尔科技有限公司 | Image write-in control method, device and electronic equipment |
| CN108921867A (en) * | 2018-06-01 | 2018-11-30 | 天津煋鸟科技有限公司 | A kind of motion detection and human body recognition method of panoramic picture |
| CN109741463B (en) * | 2019-01-02 | 2022-07-19 | 京东方科技集团股份有限公司 | Rendering method, device and equipment of virtual reality scene |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105103034A (en) * | 2013-03-25 | 2015-11-25 | 索尼电脑娱乐欧洲有限公司 | Display |
| CN105323552A (en) * | 2015-10-26 | 2016-02-10 | 北京时代拓灵科技有限公司 | Method and system for playing panoramic video |
| CN105404393A (en) * | 2015-06-30 | 2016-03-16 | 指点无限（美国）有限公司 | Low-latency virtual reality display system |
| CN105898359A (en) * | 2016-04-27 | 2016-08-24 | 乐视控股（北京）有限公司 | Virtual reality terminal and method and device for processing video thereof |
| CN105898138A (en) * | 2015-12-18 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Panoramic video play method and device |
| CN105898338A (en) * | 2015-12-18 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Panorama video play method and device |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9314691B2 (en) * | 2002-12-10 | 2016-04-19 | Sony Computer Entertainment America Llc | System and method for compressing video frames or portions thereof based on feedback information from a client device |
| US9999835B2 (en) * | 2015-02-05 | 2018-06-19 | Sony Interactive Entertainment Inc. | Motion sickness monitoring and application of supplemental sound to counteract sickness |
-
2016
- 2016-08-31 CN CN201610799113.1A patent/CN106331823B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105103034A (en) * | 2013-03-25 | 2015-11-25 | 索尼电脑娱乐欧洲有限公司 | Display |
| CN105404393A (en) * | 2015-06-30 | 2016-03-16 | 指点无限（美国）有限公司 | Low-latency virtual reality display system |
| CN105323552A (en) * | 2015-10-26 | 2016-02-10 | 北京时代拓灵科技有限公司 | Method and system for playing panoramic video |
| CN105898138A (en) * | 2015-12-18 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Panoramic video play method and device |
| CN105898338A (en) * | 2015-12-18 | 2016-08-24 | 乐视致新电子科技（天津）有限公司 | Panorama video play method and device |
| CN105898359A (en) * | 2016-04-27 | 2016-08-24 | 乐视控股（北京）有限公司 | Virtual reality terminal and method and device for processing video thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106331823A (en) | 2017-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106331823B (en) | A kind of video broadcasting method and device | |
| JP6898430B2 (en) | Continuous Time Warping and Binocular Time Warping and Methods for Virtual Reality and Augmented Reality Display Systems | |
| RU2754991C2 (en) | System of device for viewing mixed reality and method for it | |
| CN106464854B (en) | Image encodes and display | |
| CN110515452A (en) | Image processing method, device, storage medium and computer equipment | |
| CN110809149B (en) | Media compositor for computer-generated reality | |
| CN106066701B (en) | A kind of AR and VR data processing equipment and method | |
| CN110465097B (en) | Character vertical drawing display method and device in game, electronic equipment and storage medium | |
| CN105872698B (en) | Playback method, play system and virtual reality terminal | |
| CN108292489A (en) | Information processing unit and image generating method | |
| CN105939481A (en) | Interactive three-dimensional virtual reality video program recorded broadcast and live broadcast method | |
| CN106101741A (en) | Internet video live broadcasting platform is watched the method and system of panoramic video | |
| CN106598229A (en) | Virtual reality scene generation method and equipment, and virtual reality system | |
| JP2010033367A (en) | Information processor and information processing method | |
| CN106210856B (en) | The method and system of 3D panoramic video are watched on internet video live broadcasting platform | |
| CN107277485B (en) | Image display method and device based on virtual reality | |
| KR20130061538A (en) | Apparatus and method for providing contents based virtual reality | |
| JP6656382B2 (en) | Method and apparatus for processing multimedia information | |
| CN108983974B (en) | AR scene processing method, device, equipment and computer-readable storage medium | |
| EP3346375B1 (en) | Program, recording medium, content provision device, and control method | |
| WO2013041152A1 (en) | Methods to command a haptic renderer from real motion data | |
| CN107197339A (en) | Display control method, device and the head-mounted display apparatus of film barrage | |
| CN112380989B (en) | Head-mounted display equipment, data acquisition method and device thereof, and host | |
| JP2019074881A (en) | Image generation device and image generation method | |
| JP2017097854A (en) | Program, recording medium, content providing device, and control method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |