CN105872353A - System and method for implementing playback of panoramic video on mobile device - Google Patents
System and method for implementing playback of panoramic video on mobile device Download PDFInfo
- Publication number
- CN105872353A CN105872353A CN201510938176.6A CN201510938176A CN105872353A CN 105872353 A CN105872353 A CN 105872353A CN 201510938176 A CN201510938176 A CN 201510938176A CN 105872353 A CN105872353 A CN 105872353A
- Authority
- CN
- China
- Prior art keywords
- video
- panoramic video
- panoramic
- texture information
- mobile device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/62—Control of parameters via user interfaces
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
- H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
Abstract
The invention relates to a method and system for implementing playback of a panoramic video on a mobile device. The method comprises the steps of after sensing that a user acquires a selected video image by a trigger operation on a video application, creating a three-dimensional sphere in a virtual environment by using an OpenGL ES (Open Graphics Library for Embedded System); placing a camera in the three-dimensional world inside the sphere; executing a video acquiring and converting step, i.e. reading the video image at a preset speed, acquiring each frame of video image and converting the video image into texture information; executing a texture information mapping step, i.e. mapping the texture information obtained by conversion into the sphere; executing a view angle calculating step, i.e. according to a rotating angle and a camera view angle of the three-dimensional sphere, carrying out quaternion matrix calculation to obtain an actually displayed image matrix; and repeating the video acquiring and converting step, the texture information mapping step and the view angle calculating step, and acquiring a panoramic video image. Therefore, 360-degree panoramic video playback on the mobile device is implemented, and a video viewing effect is improved.
Description
Technical field
The present invention relates to technical field of digital media, realize on the mobile apparatus particularly to one playing panorama
The system and method for video.
Background technology
Along with the surge of mobile video number of users, the competition of mobile video application market is the fiercest, uses
The requirement that family is applied for mobile video is more and more higher.In existing mobile video is applied, by playing
It is also played in application interface by rectangular video picture that m3u8 or mp4 address acquisition to is regular,
But this requires that the content of above-mentioned video pictures must be the rectangular frame that a normal tiling is opened.
As is well known, panoramic video is to develop to extend in 360 panoramic techniques, will
Static panoramic pictures is converted into dynamic video image, allows video viewers have a kind of real meaning upper body to face
The sensation in its border, it is not limited by time, space and region.Briefly, traditional video may only
Seeing a face, panoramic video is it can be seen that around occurent scene.Therefore, along with panoramic video skill
Increasingly the development of art is perfect, in order to strengthen the audiovisual experience of mobile subscriber, carries out aphorama in a mobile device
Frequency is play becomes the actual demand that user is the strongest.But, in existing video playback, in order to carry out
Panoramic video is play, if the panoramic video of the sphere that do not tiles is positioned over a common player
In play out, then what user was seen would is that a undressed picture distorted, it is difficult to realize use
The audiovisual experience that family is satisfied.
Summary of the invention
In order to solve the problems referred to above, the purpose of the present invention aims to provide a kind of flat based on existing video player
Platform uses OpenGL ES technology to realize the system and method for playing panoramic video on the mobile apparatus.
A first aspect of the present invention, it is provided that a kind of method realizing playing panoramic video on the mobile apparatus, its
Including:
Panoramic video foundation step is many to produce for gathering frame of video in the range of 360 degree from different directions
Road video flowing, and described multi-path video stream is processed, generate panoramic video stream；
Central authorities rate-determining steps, instruction in described panoramic video foundation step process multi-path video stream, described entirely
Scape video creation step carries out processing to generate in real time panoramic video according to this instruction；And
Video set-up procedure, adjusts institute according in described central authorities rate-determining steps instructing the response of user operation
State panoramic video display content on mobile device display.
Preferably, according to the panoramic video player method of the present invention, described video set-up procedure farther includes:
According to user instruction, panoramic video is carried out object tracing.
Preferably, according to the panoramic video player method of the present invention, described video set-up procedure farther includes:
Panoramic video is carried out stabilization computing.
Preferably, according to the panoramic video player method of the present invention, described panoramic video foundation step is further
Including: spheroid generation step, video acquisition and switch process, texture information mapping step, visual angle calculate step
Suddenly and video pictures update step, wherein
Described spheroid generation step sensing user is by getting selected to the trigger action of Video Applications
After video pictures, OpenGL ES is used to create the three-dimensional sphere in a virtual environment；
After imaginary video camera is placed into above-mentioned three-dimensional sphere inside, described video acquisition and switch process with
Predetermined video reading speed reads described video pictures, obtains each two field picture, the video that will be read out
Image is converted to texture information；
Described texture mapping step obtains each two field picture as line in the playing process of described video pictures
Reason information, by, in the described three-dimensional sphere that texture information is mapped in virtual environment by three-dimensional conversion, passing through
Described texture information is mapped to the display of mobile device from three-dimensional world and shows by projective transformation；
Described visual angle calculation procedure carries out four according to the anglec of rotation and the camera angles of described three-dimensional sphere
Variable matrix calculates, thus obtains the picture matrix of actual displayed；
Described video pictures updates step and controls described video acquisition switch process, described texture information mapping step
Suddenly, described visual angle calculation procedure carries out iterative cycles action, continuous refreshing video picture, thus obtains 360
The panoramic video picture of degree.
Further, according to the panoramic video player method of the present invention, imaginary video camera is placed into described
Behind three-dimensional sphere inside with described video acquisition and switch process before, also include: described video camera is along with institute
The rotation stating three-dimensional sphere is spaced collection video image at regular intervals, by every frame video figure of instantaneous acquiring
As pre-processing.
Preferably, according to the panoramic video player method of the present invention, described predetermined video reading speed is every
Second more than 20 times.
Further, according to the panoramic video player method of the present invention, by view transformation, mapping transformation will
Described texture information shows after being converted to the coordinate on mobile device display.
Further, according to the panoramic video player method of the present invention, described Image semantic classification includes video
Frame carries out the process in advance of medelling, and this processes in advance and includes: the amendment color mode of frame of video, frame of video
Size or be filtered, distortion correction.
A second aspect of the present invention, it is provided that a kind of system realizing playing panoramic video on the mobile apparatus, its
Including:
Panoramic video creates device, many to produce for gathering frame of video in the range of 360 degree from different directions
Road video flowing, and described multi-path video stream is processed, generate panoramic video stream；
CCU, creates device to described panoramic video and sends the instruction of process multi-path video stream, make
Described panoramic video creates device to carry out processing to generate in real time panoramic video according to this instruction；And
Video adjusting apparatus, adjusts panorama according to described CCU to the response instruction of user operation
Video display content on mobile device display.
Preferably, according to the panoramic video Play System of the present invention, described video adjusting apparatus configures further
For panoramic video being carried out object tracing according to user instruction.
Preferably, according to the panoramic video Play System of the present invention, described video adjusting apparatus configures further
For panoramic video is carried out stabilization computing.
Preferably, according to the panoramic video Play System of the present invention, it is further that described panoramic video creates device
Including spheroid signal generating unit, video acquisition and converting unit, texture information map unit, visual angle computing unit
With video pictures updating block, wherein
Described spheroid signal generating unit sensing user is by getting selected to the trigger action of Video Applications
After video pictures, OpenGL ES is used to create the three-dimensional sphere in a virtual environment；
After imaginary video camera is placed into above-mentioned three-dimensional sphere inside, described video acquisition and converting unit with
Predetermined video reading speed reads described video pictures, obtains each two field picture, the video that will be read out
Image is converted to texture information；
Described texture map unit obtains each two field picture as line in the playing process of described video pictures
Reason information, by, in the above-mentioned three-dimensional sphere that texture information is mapped in virtual environment by three-dimensional conversion, passing through
Above-mentioned texture information is mapped to the display of mobile device from three-dimensional world and shows by projective transformation；
Described visual angle computing unit carries out four according to the anglec of rotation and the camera angles of described three-dimensional sphere
Variable matrix calculates, thus obtains the picture matrix of actual displayed；
Described video pictures updating block controls described video acquisition converting unit, described texture information maps single
Visual angle first, described computing unit carries out iterative cycles action, continuous refreshing video picture, thus obtains 360
The panoramic video picture of degree.
Preferably, according to the panoramic video Play System of the present invention, described predetermined video reading speed is every
Second more than 20 times.
Preferably, according to the panoramic video Play System of the present invention, described texture map unit is additionally operable to pass through
View transformation, mapping transformation demonstrate after described texture information is converted to the coordinate on mobile device display
Come.
Utilization by above-mentioned technological means, it is achieved that 360 degree of panoramic videos in mobile device are play, and carry
High video-see effect.
Accompanying drawing explanation
Fig. 1 is the flow chart of the method realizing playing panoramic video on the mobile apparatus of the present invention.
Fig. 2 shows that spheroid rotates the state diagram of the position relationship change with visual angle, user place, Fig. 2 (a)
Show spheroid and the position at visual angle, user place in the three-dimensional world initialized after OpenGL；Fig. 2 (b)
Show that spheroid rotates the most to the right when user uses finger to slide on the right side of screen on the left of mobile phone screen
Situation；Fig. 2 (c) shows when user uses finger to slide into below screen above mobile phone screen,
The situation that video camera correspondingly rotates up；Fig. 2 (d) show when user use two fingers respectively to
When screen sides is slided, the situation that video camera is correspondingly moved rearwards by.
Fig. 3 is the block diagram of the system realizing playing panoramic video on the mobile apparatus of the present invention.
Fig. 4 is the structural representation that the panoramic video in the panoramic video Play System of the present invention creates device.
Detailed description of the invention
At present, user commonly uses video player such as PPTV, broadcast soon, Baidu is audio-visual, windows carries
Player, MPC, MicrosoftDirectShow etc..The present invention is exactly at existing above-mentioned video
On the basis of player, use OpenGL ES technology to carry out playing panoramic video, enhance viewing video
Consumer's Experience.
OpenGL ES (OpenGL for Embedded Systems) is OpenGL 3-D graphic API
The subset of (Application Programming Interface, application programming interface), for
The embedded devices such as mobile phone, PDA and game host and design, it is possible to realize very abundant animation effect.
Generally, the panoramic video playing principles of the present invention is: the video that user triggers in mobile device should
After getting video pictures, use OpenGL ES technology to set up a spheroid, accessed is regarded
Frequently picture fits to ball interior by the way of texture is fitted, and (that is, carries out video to broadcast by video camera simultaneously
The display screen of the terminal device put) it is placed on ball interior, allow what user watched in Video Applications to regard
Frequently picture is exactly to be in the panorama at ball interior, at this moment by the finger of user or mobile device
Gyroscope rotates the angle of above-mentioned spheroid, other visual angle in thus user just can watch panoramic video.
In order to the panoramic video playing principles of the present invention and playing process being carried out cheer and bright description, hereafter
Middle the display being used for carrying out the mobile device of video playback is assumed to be video camera.
It addition, in the present invention, the scaling of the video pictures of panoramic video is by moving at above-mentioned ball interior
Before and after video camera, position is realized.
The method realizing playing panoramic video on the mobile apparatus provided by the present invention, it specifically includes that entirely
Scape video creation step, for gathering frame of video from different directions to produce multi-channel video in the range of 360 degree
Stream, and described multi-path video stream is processed, generate panoramic video stream；Central authorities' rate-determining steps, indicates
Processing multi-path video stream in described panoramic video foundation step, described panoramic video foundation step is according to this instruction
Carry out processing to generate in real time panoramic video；And video set-up procedure, according in described central authorities rate-determining steps
Response instruction to user operation adjusts described panoramic video display content on mobile device display.
Below, as a example by the panoramic video opening user's program request, it is specifically described in the present invention and how to realize
The broadcasting of panoramic video.
As it is shown in figure 1, the flow chart of the method realizing playing panoramic video on the mobile apparatus for the present invention,
It mainly includes the following steps that the operation of S11-S16.Specifically,
Step S11: sensing user is by getting selected video pictures to the trigger action of Video Applications
After, the three-dimensional sphere using OpenGL ES to create in a virtual environment (can be described as ' three-dimensional sphere establishment
Step ').
User carries out clicking operation to the Video Applications in mobile device thus obtains selected video pictures,
After system senses this information, first, OpenGL ES is initialized.
In the present invention, initialize OpenGL ES and be segmented into four steps:
The first step: call eglInitialize () and initialize egl storehouse,
Second step: use eglChooseConfig () to select suitable framebuffer,
3rd step: call eglCreateWindowSurface to create EGLSurface,
4th step: use eglCreateContext to create RenderContext (RC).
Secondly, the OpenGL ES after initializing is utilized to create the three-dimensional sphere in a virtual environment.
Specifically, user carrys out constructive geometry figure by means of OpenGL ES by drawing triangle, and leads to
Cross the limit number increasing regular polygon, thus construct the three-dimensional sphere in a virtual environment.
Step S12: after constructing three-dimensional sphere, is positioned over ball interior by the video camera in three-dimensional world
(in the present embodiment, the initial position of video camera is sphere center position, and user is observed in Video Applications
To image be exactly to be in the panorama in ball interior).
Step S13: read video pictures at a predetermined rate, obtains each two field picture, by read out
Video image is converted to texture information, and wherein said video pictures triggers mobile setting for user in step s 11
The standby upper video pictures (can be described as ' video acquisition and switch process ') accessed by Video Applications.
In the present invention, preferably read, with the speed of 30 times per second, the video pictures obtained.But the present invention's
Video reading speed is not limited to 30 times per second, it is also possible to be 60 times per second, but in order to meet video
The vision continuity of beholder, above-mentioned video reading speed at least should be at more than 20 times per second.
Step S14: the inside that the texture information changed is mapped to above-mentioned spheroid (can be described as ' texture letter
Breath mapping step ').
Wherein in above-mentioned steps S13 and step S14, the playing process of video pictures obtains each frame
Texture information, as texture information, is mapped in the three-dimensional sphere in virtual environment by image by three-dimensional conversion,
By projective transformation texture information is mapped to the display of mobile device from three-dimensional world and (such as, passes through
View transformation, mapping transformation etc. show after texture information is converted to the coordinate on display).
Step S15: the anglec of rotation and camera angles according to spheroid carry out quaternion matrix calculating, thus
Obtain the picture matrix (can be described as ' visual angle calculation procedure ') of actual displayed.
Being in the panorama in ball interior to obtain and browsing, such as, user can be by movement
The LCD screen of equipment carries out touch operation, makes above-mentioned spheroid rotate.Owing to spheroid self rotates about
Axle carries out the rotation of 360 degree, therefore user just can watch other visual angles in panoramic video.
It addition, those skilled in the art all know, universal joint deadlock be called for short the most again gimbal lock or
Universal lock, refers to, when the axle of three universal joint two of which occurs to overlap, to lose the feelings of one degree of freedom
Shape.Universal joint deadlock can cause consecutive variations on position, causes nothing between two key frames of given image
Method seamlessly transits.Therefore, in the operating process of above-mentioned playing panoramic video, in order to avoid universal joint occurs
Deadlock (Gimbal Lock), when spinning ball visual angle, spinning ball the most in the horizontal direction, along vertical
Direction rotary camera.
Fig. 2 shows that spheroid rotates the state diagram of the position relationship change with visual angle, user place, such as Fig. 2 (a)
Shown in, it is shown that initialize spheroid and the position at visual angle, user place in the three-dimensional world after OpenGL,
The most left figure is stereogram, and right figure is plane.In this Fig. 2 (a), for convenience of description, with ball
Heart position is viewpoint, and position for video camera is in the substantially sphere center position of spheroid, and the angular field of view at user place,
I.e. viewing area is with the sector region as symmetry axis of place, the visual dead ahead straight line shown in Fig. 2 (a).
Using the position at the spheroid shown in this Fig. 2 (a) and visual angle, user place as the position of original state.
As shown in Fig. 2 (b), when user uses finger to touch the display of mobile device, such as Mobile phone screen
When curtain is on the right side of left side slides into screen, in response to this touch operation of user, above-mentioned spheroid is to correspondingly right side
Rotating to an angle, in this Fig. 2 (b), left figure is stereogram, and right figure is plane.Now, such as right figure
Shown in, along with the variation of camera position, the angular field of view of user becomes with the visual dead ahead of user in scheming
Place straight line is the sector region of original position.
As shown in Fig. 2 (c), when user uses finger to touch the display of mobile device, such as from mobile phone
When sliding into below screen above screen, video camera rotates some angles, this Fig. 2 (c) to correspondingly upside
Middle left figure is stereogram, and right figure is plane.Now, as shown at right, the angular field of view of user become with
The sector region of line centered by the visual dead ahead of user in figure.
As shown in Fig. 2 (d), when user uses finger to touch the display of mobile device, such as, use two
Finger is respectively when screen sides is slided, and video camera is correspondingly moved rearwards by, left in this Fig. 2 (d)
Figure is stereogram, and right figure is plane.Now, as shown at right, the angular field of view of user is (i.e., visually
Region) become in scheming the sector region of line centered by the visual dead ahead of user, but, this viewing area
The center of circle that place is fan-shaped, i.e. camera position offsets downward some distances, thus can realize panoramic video
The amplification of picture and reducing.
In the rotary course of above-mentioned spheroid, video camera is spaced collection at regular intervals along with the rotation of spheroid
Video image, pre-processes every frame video image of instantaneous acquiring, synthesizes full-view video image in real time,
And carry out calculating, splicing.Wherein said Image semantic classification refers to such as frame of video be carried out medelling in advance
Process, such as revise the color mode of frame of video, the size of frame of video or be filtered, distortion correction etc..
Preferably, each frame of the panoramic video of the present invention is editable, and can generate panorama
In real time the frame of panoramic video is edited while video.It is likewise possible to during gathering frame of video
And response comes from the control instruction of user in real time after processing frame of video.
Then, the actual anglec of rotation and camera angles according to spheroid carry out Quaternion Matrix calculating, from
And obtain the picture matrix of actual displayed.
As it is known by the man skilled in the art that three angle system cannot show the rotation of any axle, as long as at the beginning
Rotating, object itself i.e. loses the independence to any axle.And quaternary number is the most commutative extension of plural number,
If the set of quaternary number considers into multidimensional real number space, quaternary number just represents a space-time,
It is two-dimensional space relative to plural number.In order to simply be easily removed the key elements such as spheroid rotary shaft and the anglec of rotation
Feature, the present invention uses quaternary number to be indicated element characteristic, and according to the actual anglec of rotation of spheroid with
And camera angles carries out Quaternion Matrix calculating, thus obtain the video pictures of actual displayed.
Step S16: the operation of repeat the above steps S13-S15, continuous refreshing video picture, thus obtain
The panoramic video display picture of 360 degree.
By the operation of above step S11-S16, it becomes possible to realize playing in a mobile device the panorama of smoothness
Video, it is thus achieved that satisfied audiovisual experience.
Another embodiment of the present invention for provide a kind of realize playing panoramic video on the mobile apparatus be
System.As it is shown on figure 3, the block diagram of the system realizing playing panoramic video on the mobile apparatus for the present invention.
The panoramic video Play System 10 of the present invention includes:
Panoramic video creates device 11, for gathering frame of video in the range of 360 ° from different directions to produce
Multi-path video stream, and described multi-path video stream is processed, generate panoramic video stream；
CCU 12, creates device 11 to panoramic video and sends the instruction of process multi-path video stream,
Make above-mentioned panoramic video create device 11 to carry out processing to generate in real time panoramic video according to this instruction；And
Video adjusting apparatus 13, instructs the response of user operation according to described CCU (12)
Adjust panoramic video display content on mobile device display.
Specifically, mobile device can by himself configuration display etc. receive user adjust display in
The adjustment instruction held, now, the adjustment instruction that described video adjusting apparatus 13 inputs according to user comes corresponding
Ground adjusts the display content of mobile device display.Such as, the brightness of video, contrast, tone etc. are adjusted
Deng or change the size of panoramic video, viewing angle etc..As described above, panoramic video includes 360 °
In the range of scene information, user can freely select desired by it viewing according to being actually needed of self
Scene.
Alternatively, above-mentioned video adjusting apparatus 103 is further configured to enter panoramic video according to user instruction
Row object tracing.Specifically,
In the present invention, mobile device can receive user tracking pair by the display etc. of himself configuration
The instruction of elephant.Now, first above-mentioned video adjusting apparatus 103 can be waited to follow the trail of according to the command detection of user
Object, i.e. destination object.Concrete detection process is exactly to feel emerging from the sequence of frames of video of panoramic video
The region of interest, i.e. targeted object region extracts from background image, is formed after To Template, above-mentioned regards
Frequency modulation engagement positions 103 find in sequence of frames of video the position of the image most like with above-mentioned To Template with
Tracing object.
Preferably, above-mentioned video adjusting apparatus 103 can use based on characteristics of objects point, object template or right
As the object tracking method of movable information carrys out tracing object.
Alternatively, above-mentioned video adjusting apparatus 103 can be further configured to panoramic video is carried out stabilization fortune
Calculate.
Preferably, the above-mentioned process to described multi-path video stream can be to utilize parallel computation and splicing multichannel to regard
The synchronized video frames of frequency stream generates panoramic video in real time.
As shown in Figure 4, the structure of device is created for the panoramic video in the panoramic video Play System of the present invention
Schematic diagram.Above-mentioned panoramic video creates device 11 and farther includes spheroid signal generating unit 111, video acquisition
With converting unit 112, texture map unit 113, visual angle computing unit 114 and video pictures updating block
115, wherein
It is selected by getting the trigger action of Video Applications that above-mentioned spheroid signal generating unit 111 senses user
After fixed video pictures, OpenGL ES is used to create the three-dimensional sphere in a virtual environment.
After imaginary video camera is placed into above-mentioned three-dimensional sphere inside, above-mentioned video acquisition and converting unit
112 read video at a predetermined rate, obtain each two field picture, are converted to by the video image read out
Texture information, wherein said video pictures is regarding accessed by the Video Applications that user triggers in mobile device
Frequently picture.
As described above, in the present invention, preferably read video pictures with the speed of 30 times per second.But this
Bright video reading speed is not limited to 30 times per second, it is also possible to be 60 times per second.But in order to meet
The vision continuity of video viewers, above-mentioned video reading speed should be at more than 20 times per second.
Above-mentioned texture information map unit 113 obtains each two field picture in the playing process of video pictures and makees
For texture information, change in the above-mentioned three-dimensional sphere that texture information is mapped in virtual environment by three-dimensional,
By projective transformation above-mentioned texture information is mapped to the display of mobile device from three-dimensional world and demonstrates
Come.
Above-mentioned visual angle computing unit 114 carries out quaternary square according to the anglec of rotation and the camera angles of spheroid
Battle array calculates, thus obtains the picture matrix of actual displayed.
Above-mentioned video pictures updating block 115 controls above-mentioned video acquisition and converting unit 112, above-mentioned texture
Information MAP unit 113, above-mentioned visual angle computing unit 114 carry out iterative cycles action, continuous refreshing video
Picture, thus obtain the panoramic video picture of 360 degree.
The present invention uses panoramic video player method as above and Play System, and user can be by means of shifting
The display screen of dynamic equipment is immersed in the world of dynamic panorama of 360 degree, enjoys visual experience on the spot in person,
Either real-time live broadcast or program request, user can choose visual angle the most continuously, it is achieved outstanding interaction
Experience.
Those of ordinary skill in the field it is understood that the discussion of any of the above embodiment is exemplary only,
It is not intended to imply that the scope of the present disclosure (including claim) is limited to these examples；Think of in the present invention
Under road, can also be combined between the technical characteristic in above example or different embodiment, step can
With with random order realize, and exist the present invention as above different aspect many other change, for
Simple and clear they do not provide in the details.Therefore, all within the spirit and principles in the present invention, done
Any omission, amendment, equivalent, improvement etc., should be included within the scope of the present invention.
Claims (14)
1. the method realizing playing panoramic video on the mobile apparatus, it is characterised in that including:
Panoramic video foundation step, gathers frame of video in the range of 360 degree from different directions and regards to produce multichannel
Frequency stream, and described multi-path video stream is processed, generate panoramic video stream；
Central authorities rate-determining steps, instruction in described panoramic video foundation step process multi-path video stream, described entirely
Scape video creation step carries out processing to generate in real time panoramic video according to this instruction；And
Video set-up procedure, adjusts institute according in described central authorities rate-determining steps instructing the response of user operation
State panoramic video display content on mobile device display.
Panoramic video player method the most according to claim 1, it is characterised in that described video adjusts
Step farther includes: according to user instruction, panoramic video is carried out object tracing.
Panoramic video player method the most according to claim 1, it is characterised in that described video adjusts
Step farther includes: panoramic video is carried out stabilization computing.
4. according to the panoramic video player method described in any one in claim 1-3, it is characterised in that
Described panoramic video foundation step farther includes: spheroid generation step, video acquisition and switch process, line
Reason information MAP step, visual angle calculation procedure and video pictures update step, wherein
Described spheroid generation step sensing user is by getting selected to the trigger action of Video Applications
After video pictures, OpenGL ES is used to create the three-dimensional sphere in a virtual environment；
After imaginary video camera is placed into above-mentioned three-dimensional sphere inside, described video acquisition and switch process with
Predetermined video reading speed reads described video pictures, obtains each two field picture, the video that will be read out
Image is converted to texture information；
Described texture mapping step obtains each two field picture as line in the playing process of described video pictures
Reason information, by, in the described three-dimensional sphere that texture information is mapped in virtual environment by three-dimensional conversion, passing through
Described texture information is mapped to the display of mobile device from three-dimensional world and shows by projective transformation；
Described visual angle calculation procedure carries out four according to the anglec of rotation and the camera angles of described three-dimensional sphere
Variable matrix calculates, thus obtains the picture matrix of actual displayed；
Described video pictures updates step and controls described video acquisition switch process, described texture information mapping step
Suddenly, described visual angle calculation procedure carries out iterative cycles action, continuous refreshing video picture, thus obtains 360
The panoramic video picture of degree.
Panoramic video player method the most according to claim 4, it is characterised in that by imaginary shooting
Machine be placed into behind described three-dimensional sphere inside with described video acquisition and switch process before, also include:
Described video camera is spaced collection video image at regular intervals along with the rotation of described three-dimensional sphere, will
Every frame video image of instantaneous acquiring pre-processes.
6. according to the panoramic video player method described in claim 4 or 5, it is characterised in that described predetermined
Video reading speed be more than 20 times per second.
7. according to the panoramic video player method described in claim 4 or 5, it is characterised in that at described line
In reason mapping step, also include:
By view transformation, mapping transformation described texture information is converted to the coordinate on mobile device display
After show.
Panoramic video player method the most according to claim 5, it is characterised in that described video image
Pretreatment include frame of video is carried out medelling process in advance, this processes in advance and includes: amendment frame of video
Color mode, the size of frame of video or be filtered, distortion correction.
9. the system realizing playing panoramic video on the mobile apparatus, it is characterised in that including:
Panoramic video creates device, many to produce for gathering frame of video in the range of 360 degree from different directions
Road video flowing, and described multi-path video stream is processed, generate panoramic video stream；
CCU, creates device to described panoramic video and sends the instruction of process multi-path video stream, make
Described panoramic video creates device to carry out processing to generate in real time panoramic video according to this instruction；And
Video adjusting apparatus, adjusts panorama according to described CCU to the response instruction of user operation
Video display content on mobile device display.
Panoramic video Play System the most according to claim 9, it is characterised in that described video is adjusted
Engagement positions is further configured to, according to user instruction, panoramic video is carried out object tracing.
11. panoramic video Play Systems according to claim 9, it is characterised in that described video is adjusted
Engagement positions is further configured to panoramic video is carried out stabilization computing.
12. according to the panoramic video Play System described in any one in claim 9-11, it is characterised in that
Described panoramic video creates device and farther includes spheroid signal generating unit, video acquisition and converting unit, texture
Information MAP unit, visual angle computing unit and video pictures updating block, wherein
Described spheroid signal generating unit sensing user is by getting selected to the trigger action of Video Applications
After video pictures, OpenGL ES is used to create the three-dimensional sphere in a virtual environment；
After imaginary video camera is placed into above-mentioned three-dimensional sphere inside, described video acquisition and converting unit with
Predetermined video reading speed reads described video pictures, obtains each two field picture, the video that will be read out
Image is converted to texture information；
Described texture map unit obtains each two field picture as line in the playing process of described video pictures
Reason information, by, in the described three-dimensional sphere that texture information is mapped in virtual environment by three-dimensional conversion, passing through
Described texture information is mapped to the display of mobile device from three-dimensional world and shows by projective transformation；
Described visual angle computing unit carries out four according to the anglec of rotation and the camera angles of described three-dimensional sphere
Variable matrix calculates, thus obtains the picture matrix of actual displayed；
Described video pictures updating block controls described video acquisition converting unit, described texture information maps single
Visual angle first, described computing unit carries out iterative cycles action, continuous refreshing video picture, thus obtains 360
The panoramic video picture of degree.
13. panoramic video Play Systems according to claim 12, it is characterised in that described predetermined
Video reading speed is more than 20 times per second.
14. panoramic video Play Systems according to claim 12, it is characterised in that described texture reflects
Penetrate unit to be additionally operable to, by view transformation, mapping transformation, described texture information is converted to mobile device display
On coordinate after show.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510938176.6A CN105872353A (en) | 2015-12-15 | 2015-12-15 | System and method for implementing playback of panoramic video on mobile device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510938176.6A CN105872353A (en) | 2015-12-15 | 2015-12-15 | System and method for implementing playback of panoramic video on mobile device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105872353A true CN105872353A (en) | 2016-08-17 |
Family
ID=56623713
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510938176.6A Pending CN105872353A (en) | 2015-12-15 | 2015-12-15 | System and method for implementing playback of panoramic video on mobile device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105872353A (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106412480A (en) * | 2016-08-24 | 2017-02-15 | 乐视控股（北京）有限公司 | Method and device for displaying video picture of mobile terminal |
| CN106598428A (en) * | 2016-11-29 | 2017-04-26 | 宇龙计算机通信科技(深圳)有限公司 | Method and system for playing panoramic video, and terminal equipment |
| CN106610721A (en) * | 2016-12-16 | 2017-05-03 | 飞狐信息技术（天津）有限公司 | Method and apparatus for realizing simultaneous response to gyroscope and gesture dragging interaction, and mobile phone |
| CN106658220A (en) * | 2016-11-11 | 2017-05-10 | 理光图像技术（上海）有限公司 | Subtitle creating device, demonstration module and subtitle creating demonstration system |
| CN106648091A (en) * | 2016-12-16 | 2017-05-10 | 飞狐信息技术（天津）有限公司 | Method and device of simultaneously and accurately responding to gyro and gesture dragging interaction and mobile phone |
| CN106982298A (en) * | 2017-04-01 | 2017-07-25 | 小派科技（上海）有限责任公司 | A kind of implementation method of virtual reality mobile terminal and virtual reality mobile terminal |
| CN107146274A (en) * | 2017-05-05 | 2017-09-08 | 上海兆芯集成电路有限公司 | Image data processing system, texture mapping compression and the method for producing panoramic video |
| CN107547939A (en) * | 2017-08-18 | 2018-01-05 | 深圳岚锋创视网络科技有限公司 | Method, system and the portable terminal of panoramic video file clip |
| CN107888970A (en) * | 2017-11-29 | 2018-04-06 | 天津聚飞创新科技有限公司 | Method for processing video frequency, device, embedded device and storage medium |
| CN107959844A (en) * | 2016-10-14 | 2018-04-24 | 安华高科技通用Ip(新加坡)公司 | 360 degree of video captures and playback |
| CN108366243A (en) * | 2018-01-23 | 2018-08-03 | 微幻科技(北京)有限公司 | A kind of video jitter removing method and device |
| CN108574876A (en) * | 2017-03-10 | 2018-09-25 | 优估（上海）信息科技有限公司 | A kind of anti-fluttering method, device and the mobile terminal of mobile terminal 3D display |
| CN110276822A (en) * | 2018-03-13 | 2019-09-24 | 英属开曼群岛商玩美股份有限公司 | It is implemented in the system for calculating equipment, method and storage media |
| CN110546961A (en) * | 2019-06-25 | 2019-12-06 | 北京小米移动软件有限公司 | Panoramic media playing method and device and computer readable storage medium |
| CN110662099A (en) * | 2018-06-28 | 2020-01-07 | 北京京东尚科信息技术有限公司 | Method and device for displaying bullet screen |
| CN111093036A (en) * | 2020-01-17 | 2020-05-01 | 福州大学 | VR panoramic video interactive playing method based on limb recognition |
| CN112437276A (en) * | 2020-11-20 | 2021-03-02 | 埃洛克航空科技（北京）有限公司 | WebGL-based three-dimensional video fusion method and system |
| CN112788317A (en) * | 2020-12-30 | 2021-05-11 | 惠州Tcl移动通信有限公司 | Method, system, storage medium and playing device for playing panoramic video |
| CN112804514A (en) * | 2020-12-31 | 2021-05-14 | 百视通网络电视技术发展有限责任公司 | VR panoramic video display interaction method, medium and equipment |
| CN114390222A (en) * | 2022-03-24 | 2022-04-22 | 北京唱吧科技股份有限公司 | Switching method and device suitable for 180-degree panoramic video and storage medium |
| CN114630037B (en) * | 2022-03-22 | 2024-02-02 | 北京有竹居网络技术有限公司 | Shooting device, control method and device thereof, readable storage medium and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050128286A1 (en) * | 2003-12-11 | 2005-06-16 | Angus Richards | VTV system |
| CN101561988A (en) * | 2009-05-20 | 2009-10-21 | 北京水晶石数字科技有限公司 | Method for exhibiting panoramagram |
| US20100231687A1 (en) * | 2009-03-16 | 2010-09-16 | Chase Real Estate Services Corporation | System and method for capturing, combining and displaying 360-degree "panoramic" or "spherical" digital pictures, images and/or videos, along with traditional directional digital images and videos of a site, including a site audit, or a location, building complex, room, object or event |
| CN101852980A (en) * | 2010-06-09 | 2010-10-06 | 长春理工大学 | Method for interactively playing panoramic video stream on CAVE projection system |
| CN101877139A (en) * | 2009-04-30 | 2010-11-03 | 爱国者全景(北京)网络科技发展有限公司 | Method and system for realizing spacial hot spots in three-dimensional video panorama |
| TWI434129B (en) * | 2010-12-22 | 2014-04-11 | Univ Nat Taipei Technology | System and device for displaying spherical panorama image |
| CN104184961A (en) * | 2013-05-22 | 2014-12-03 | 辉达公司 | Mobile device and system used for generating panoramic video |
| CN105120194A (en) * | 2015-08-06 | 2015-12-02 | 佛山六滴电子科技有限公司 | Method of recording panoramic video and apparatus thereof |
-
2015
- 2015-12-15 CN CN201510938176.6A patent/CN105872353A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050128286A1 (en) * | 2003-12-11 | 2005-06-16 | Angus Richards | VTV system |
| US20100231687A1 (en) * | 2009-03-16 | 2010-09-16 | Chase Real Estate Services Corporation | System and method for capturing, combining and displaying 360-degree "panoramic" or "spherical" digital pictures, images and/or videos, along with traditional directional digital images and videos of a site, including a site audit, or a location, building complex, room, object or event |
| CN101877139A (en) * | 2009-04-30 | 2010-11-03 | 爱国者全景(北京)网络科技发展有限公司 | Method and system for realizing spacial hot spots in three-dimensional video panorama |
| CN101561988A (en) * | 2009-05-20 | 2009-10-21 | 北京水晶石数字科技有限公司 | Method for exhibiting panoramagram |
| CN101852980A (en) * | 2010-06-09 | 2010-10-06 | 长春理工大学 | Method for interactively playing panoramic video stream on CAVE projection system |
| TWI434129B (en) * | 2010-12-22 | 2014-04-11 | Univ Nat Taipei Technology | System and device for displaying spherical panorama image |
| CN104184961A (en) * | 2013-05-22 | 2014-12-03 | 辉达公司 | Mobile device and system used for generating panoramic video |
| CN105120194A (en) * | 2015-08-06 | 2015-12-02 | 佛山六滴电子科技有限公司 | Method of recording panoramic video and apparatus thereof |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106412480A (en) * | 2016-08-24 | 2017-02-15 | 乐视控股（北京）有限公司 | Method and device for displaying video picture of mobile terminal |
| CN107959844A (en) * | 2016-10-14 | 2018-04-24 | 安华高科技通用Ip(新加坡)公司 | 360 degree of video captures and playback |
| CN106658220A (en) * | 2016-11-11 | 2017-05-10 | 理光图像技术（上海）有限公司 | Subtitle creating device, demonstration module and subtitle creating demonstration system |
| CN106598428A (en) * | 2016-11-29 | 2017-04-26 | 宇龙计算机通信科技(深圳)有限公司 | Method and system for playing panoramic video, and terminal equipment |
| CN106648091A (en) * | 2016-12-16 | 2017-05-10 | 飞狐信息技术（天津）有限公司 | Method and device of simultaneously and accurately responding to gyro and gesture dragging interaction and mobile phone |
| CN106610721A (en) * | 2016-12-16 | 2017-05-03 | 飞狐信息技术（天津）有限公司 | Method and apparatus for realizing simultaneous response to gyroscope and gesture dragging interaction, and mobile phone |
| CN108574876A (en) * | 2017-03-10 | 2018-09-25 | 优估（上海）信息科技有限公司 | A kind of anti-fluttering method, device and the mobile terminal of mobile terminal 3D display |
| CN106982298A (en) * | 2017-04-01 | 2017-07-25 | 小派科技（上海）有限责任公司 | A kind of implementation method of virtual reality mobile terminal and virtual reality mobile terminal |
| CN107146274A (en) * | 2017-05-05 | 2017-09-08 | 上海兆芯集成电路有限公司 | Image data processing system, texture mapping compression and the method for producing panoramic video |
| CN107547939A (en) * | 2017-08-18 | 2018-01-05 | 深圳岚锋创视网络科技有限公司 | Method, system and the portable terminal of panoramic video file clip |
| CN107888970A (en) * | 2017-11-29 | 2018-04-06 | 天津聚飞创新科技有限公司 | Method for processing video frequency, device, embedded device and storage medium |
| CN108366243B (en) * | 2018-01-23 | 2019-10-29 | 微幻科技(北京)有限公司 | A kind of video jitter removing method and device |
| CN108366243A (en) * | 2018-01-23 | 2018-08-03 | 微幻科技(北京)有限公司 | A kind of video jitter removing method and device |
| CN110276822A (en) * | 2018-03-13 | 2019-09-24 | 英属开曼群岛商玩美股份有限公司 | It is implemented in the system for calculating equipment, method and storage media |
| CN110662099A (en) * | 2018-06-28 | 2020-01-07 | 北京京东尚科信息技术有限公司 | Method and device for displaying bullet screen |
| CN110662099B (en) * | 2018-06-28 | 2023-05-02 | 北京京东尚科信息技术有限公司 | Method and device for displaying bullet screen |
| CN110546961B (en) * | 2019-06-25 | 2021-02-09 | 北京小米移动软件有限公司 | Panoramic media playing method and device and computer readable storage medium |
| CN110546961A (en) * | 2019-06-25 | 2019-12-06 | 北京小米移动软件有限公司 | Panoramic media playing method and device and computer readable storage medium |
| CN111093036A (en) * | 2020-01-17 | 2020-05-01 | 福州大学 | VR panoramic video interactive playing method based on limb recognition |
| CN112437276B (en) * | 2020-11-20 | 2023-04-07 | 埃洛克航空科技（北京）有限公司 | WebGL-based three-dimensional video fusion method and system |
| CN112437276A (en) * | 2020-11-20 | 2021-03-02 | 埃洛克航空科技（北京）有限公司 | WebGL-based three-dimensional video fusion method and system |
| CN112788317A (en) * | 2020-12-30 | 2021-05-11 | 惠州Tcl移动通信有限公司 | Method, system, storage medium and playing device for playing panoramic video |
| WO2022141781A1 (en) * | 2020-12-30 | 2022-07-07 | 惠州Tcl移动通信有限公司 | Method and system for playing panoramic video, storage medium, and playback device |
| CN112804514A (en) * | 2020-12-31 | 2021-05-14 | 百视通网络电视技术发展有限责任公司 | VR panoramic video display interaction method, medium and equipment |
| CN114630037B (en) * | 2022-03-22 | 2024-02-02 | 北京有竹居网络技术有限公司 | Shooting device, control method and device thereof, readable storage medium and electronic equipment |
| CN114390222B (en) * | 2022-03-24 | 2022-07-08 | 北京唱吧科技股份有限公司 | Switching method and device suitable for 180-degree panoramic video and storage medium |
| CN114390222A (en) * | 2022-03-24 | 2022-04-22 | 北京唱吧科技股份有限公司 | Switching method and device suitable for 180-degree panoramic video and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105872353A (en) | System and method for implementing playback of panoramic video on mobile device | |
| US20200288098A1 (en) | Method, apparatus, medium, terminal, and device for multi-angle free-perspective interaction | |
| CN106296783B (en) | A kind of space representation method of combination space overall situation 3D view and panoramic pictures | |
| US10659685B2 (en) | Control of viewing angles for 360-degree video playback | |
| US20080246759A1 (en) | Automatic Scene Modeling for the 3D Camera and 3D Video | |
| US20160203646A1 (en) | System and method for providing virtual reality content | |
| CN108292489A (en) | Information processing unit and image generating method | |
| WO2012046371A1 (en) | Image display device, and image display method | |
| CN104915979A (en) | System capable of realizing immersive virtual reality across mobile platforms | |
| WO2022002181A1 (en) | Free viewpoint video reconstruction method and playing processing method, and device and storage medium | |
| Coffin et al. | Enhancing classroom and distance learning through augmented reality | |
| CN109189302B (en) | Control method and device of AR virtual model | |
| CN106028115A (en) | Video playing method and device | |
| WO2017128887A1 (en) | Method and system for corrected 3d display of panoramic image and device | |
| CN104740874A (en) | Method and system for playing videos in two-dimension game scene | |
| CN108174174A (en) | VR image display methods, device and terminal | |
| CN105939497A (en) | Media streaming system and media streaming method | |
| CN107197135B (en) | Video generation method and video generation device | |
| CN110290409A (en) | Data processing method, VR equipment and system | |
| WO2020206647A1 (en) | Method and apparatus for controlling, by means of following motion of user, playing of video content | |
| WO2022001865A1 (en) | Depth map and video processing and reconstruction methods and apparatuses, device, and storage medium | |
| TWI755636B (en) | Method and program for playing virtual reality image | |
| EP3542877A1 (en) | Optimized content sharing interaction using a mixed reality environment | |
| WO2019241712A1 (en) | Augmented reality wall with combined viewer and camera tracking | |
| CN113253843B (en) | Indoor virtual roaming realization method and realization system based on panorama |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20160817 |