CN106127680A - A kind of 720 degree of panoramic video fast browsing methods - Google Patents

Abstract

The invention discloses a kind of 720 degree of panoramic video fast browsing methods, first pass through back projection method 720 degree of full-view video images are reconstructed, obtain view corresponding on each direction of visual lines of sphere view space, then camera lens length is judged by the absolute brightness frame difference of adjacent image frame in calculating video sequence, then key frame is extracted, it is achieved panoramic video fast browsing.The present invention can be quickly generated virtual scene see-through view on different sight direction, and analogue camera rotates and zoom motion effectively on all directions view, improve the surfing of virtual scene, it is possible to well meet this specific application area of virtual reality system.

Description

A kind of 720 degree of panoramic video fast browsing methods

Technical field

The invention belongs to technical field of image processing, relate to video panorama image and process, refer in particular to a kind of 720 degree of panoramic videos Fast browsing method.

Background technology

Along with the development of information technology, the scene information obtaining wide viewing angle scope is required more and more higher by people, and tradition is taken the photograph Image method can only obtain the picture frame of limited angular range, and image mosaic technology produces for solving this difficult problem and sends out rapidly Exhibition.It is the complete image by two width of phase mutual overlay information or several pictures are spliced into a ultra-wide visual angle, to reach To reducing picture redundancy and obtaining the purpose of more wide viewing angle information, wherein the generation of panoramic picture is also image mosaic technology One typical case's application.

720 degree of panoramic videos are a kind of sequence of video images based on Sphere Measurement Model, can realize horizontal 360-degree with vertical Looking around of 360 degree of visual angle directions browses.When browsing, need according to current gaze direction and FOV (Field of View) spherical video figure As carrying out back project, to obtain the plane perspective image meeting human eye vision custom.In this way, can be the most real Existing analogue camera rotary motion and zoom motion, change the ken.

Need to take a substantial amount of time and energy for having access to and play back massive video data, traditional dilatory browsing method Easily ignoring the unexpected abnormality event in the short time, the search of long-time video data is unfavorable for the extraction of effective information.Therefore Needing panoramic video has been carried out further process, it is achieved the fast browsing to panoramic video, its core work is to original The extraction of critical sequences in the segmentation of video and video.

Currently, Video segmentation and extraction method of key frame are broadly divided into four classes:

One is simple generating algorithm, by video sequence being carried out the method waiting time uniform sampling to extract key frame, But this method is due to amount of video information change difference in the short time, easily occur that key-frame extraction too much or represents asking of deficiency Topic；

Two is the generation method of view-based access control model information, according to the visual informations such as the color in video, shape, texture, application Various video processing technique carry out the operations such as scene clustering, shot detection, key-frame extraction, ultimately generate video skimming, this The method of view-based access control model feature is significantly improved in simple generating algorithm, but have ignored the audio frequency in original video, word The information such as curtain；

Three is the generation method merging multiple features, as used the appearance of VIP in face recognition technology detection news, Utilize the wonderful etc. in audio signal processing technique detection sports video, in conjunction with feature and other image procossing skills of video self The art multiple Feature Fusion to video, algorithm process process is more complicated；

Four is the generation method semantic based on video syntax, seeks between camera lens and camera lens, knot between scene and scene Structure rule, forms video frequency abstract based on this.

In sum, for different video type and purposes, different to video fast browsing processing method, current panorama It is empty that video technique is widely used in the networks such as tourist attractions, house property house, car show, leisure club, urban architecture planning In intending showing, these video scenes mainly reach preferably to publicize representing panorama to a kind of experience on the spot in person of people and perfection Purpose.

Summary of the invention

The present invention proposes a kind of 720 degree of panoramic video fast browsing methods, and it achieves level 360 by back projection method Observe video under degree and vertical 360 degree of omnibearing angels, and according to the difference of taking lens length under video difference scene, extract Key frame, forms video summary, reaches the purpose of fast browsing.

A kind of 720 degree of panoramic video fast browsing methods, comprise the following steps:

720 degree of full-view video images are reconstructed by S1 first with Inverse Projection, obtain sphere view space each View sequence corresponding on direction of visual lines.

S2, judges camera lens length by calculating the absolute brightness frame difference of adjacent image frame in video sequence, then extracts Key frame, it is achieved panoramic video fast browsing.

Wherein, S1 comprises the following steps:

S1.1, completes the splicing of 720 degree of panoramic pictures, and sets up centered by the centre of sphere based on sphere view space model Two coordinate systems, represent world coordinate system XYZ and camera coordinate system xyz respectively；Wherein camera coordinate system xyz is by the world Coordinate system XYZ X-axis rotation alpha angle in world coordinate system, obtains further around the Y-axis rotation β angle in world coordinate system.

The method of the splicing completing 720 degree of panoramic pictures based on sphere view space model in S1.1 is: according to photographing unit Straight line vertical with image transverse axis in the image that straight line parallel with y-axis in coordinate system xyz generates according to sphere parameters transform It is still this character of vertical line, multiple real scene images of fish eye lens shooting is carried out rotation transformation correction, obtains every realistic picture As the azimuth information at view space of upper pixel, utilize azimuth information that multiple images are spliced, eliminate real scene image Between duplicate message that may be present, finally project on a sphere, store with the form of Spherical Panorama Image.

S1.2, by unified for the basic measuring unit of pixel under Two coordinate system in S1.1, then to calculate using pixel as base Each pixel is i.e. estimated the viewpoint pixel focal length to view plane under camera coordinate system by the pixel focal length of this unit f。

In S1.2, if image S is the Spherical Panorama Image that a splicing is complete, Q is any one on Spherical Panorama Image S Pixel, image coordinate is；J is the view needing to generate, and some P is that the some Q on Spherical Panorama Image is corresponding on view J Point, its image coordinate is；F represents pixel focal length, and f uses camera lens to estimate according to shooting real scene image.

The method of estimation of the pixel focal distance f of wide-angle lens or standard lens is: sets camera level and rotates a circle captured Real scene image n open, then the horizontal view angle of photographing unit is 360/n, and real scene image width is W, permissible according to trigonometric function relation The pixel focal length estimation formulas drawing common lens is:

F=W/ (2tan (180/n)).

The method of estimation of fish-eye pixel focal distance f is: the width of image after the dark border of fish eye images being removed Degree is designated as W, and the most fish-eye pixel focal length estimation formulas is: f=W/ φ, wherein φ is fish-eye horizontal view.

S1.3, sets up the coordinate of X-Y scheme picture point and turning of the three-dimensional parameter coordinate points corresponding to sphere by pixel focal distance f Change relational expression, further according to world coordinate system XYZ X-axis rotation alpha angle in world coordinate system, further around the Y in world coordinate system During axle rotation β angle, along with the rotation of coordinate axes, pixel expression on each coordinate components also occurs to become accordingly (after coordinate system rotation, the correspondence position of each pixel needs again to represent under new coordinate system, and coordinate components is each leisure in change Respective components on tri-coordinate axess of x, y, z), this change can be closed with trigonometric function and be tied up to represent on respective coordinate components Come, thus obtain the transformation matrix H of corresponding point under Two coordinate system.

S1.4, is set up counter-transformation function by transformation matrix H, finds out on panoramic picture arbitrfary point to each view of spherical space The corresponding relation formula of point, calculates each point coordinates, obtains view corresponding on each direction of visual lines of view space.

In S1.3, set up coordinate and the three-dimensional parameter coordinate points corresponding to sphere of X-Y scheme picture point by pixel focal distance f Transforming relationship formula, as follows:

$\left\{\begin{array}{c}u=fcos(\frac{\mathrm{\π}}{2}-\frac{y}{f})cos\left(\frac{x}{f}\right)\\ v=fsin(\frac{\mathrm{\π}}{2}-\frac{y}{f})\\ w=fcos(\frac{\mathrm{\π}}{2}-\frac{y}{f})sin\left(\frac{x}{f}\right)\end{array}\right.---\left(1\right)$

Calculating the transformation matrix H of corresponding point under Two coordinate system, expression formula is as follows:

$H=\left[\begin{array}{ccc}1& 0& 0\\ 0& cos\mathrm{\α}& sin\mathrm{\α}\\ 0& -sin\mathrm{\α}& cos\mathrm{\α}\end{array}\right]\left[\begin{array}{ccc}cos\mathrm{\β}& 0& -\sin \mathrm{\β}\\ 0& 1& 0\\ sin\mathrm{\β}& 0& cos\mathrm{\β}\end{array}\right]---\left(2\right)$

In S1.4, from the formula (1) in S1.3 and the point under (2), coordinate system XYZ, correspondence under coordinate system xyz Coordinate is.

In known video, the width of captured real scene image is W, and height is H, sets up any point Q on Spherical Panorama Image The functional relation of the some P corresponding on view J with a Q, and utilize formula (3) to calculate the coordinate of each corresponding point, obtain viewpoint View corresponding on each direction of visual lines in space.

$\left\{\begin{array}{c}x=\frac{W}{2}-\frac{f\cos (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\cos (\frac{x^{\′}}{f}+\mathrm{\β})}{\cos (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\cos \mathrm{\α}\sin (\frac{x^{\′}}{f}+\mathrm{\β})-\sin (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\sin \mathrm{\α}}\\ y=\frac{H}{2}-\frac{f\sin (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\cos \mathrm{\α}+f\cos (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\sin \mathrm{\α}\sin (\frac{x^{\′}}{f}+\mathrm{\β})}{\cos (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\cos \mathrm{\α}\sin (\frac{x^{\′}}{f}+\mathrm{\β})-\sin (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\sin \mathrm{\α}}\end{array}\right.---\left(3\right)$

The S2 of the present invention comprises the following steps:

S2.1, carries out structuring process to panoramic video sequences, and panoramic video is corresponding on each direction of visual lines Being all the video sequence of one group of view composition, video sequence step S1 obtained is according to being projected in regarding on different directions visual angle Figure frame sequence is classified, and obtains the video sequence group that can individually browse on multiple visual angle；

S2.2, splits the video sequence group of each view directions respectively, calculates adjacent image frame in video sequence Absolute brightness frame is poor, it is judged that the switching node of video lens, is multiple camera lens fragments by video signal process；

S2.3, calculates the quantity of motion summation of each camera lens fragment, sets quantity of motion and weighs thresholding, sentences according to shot durations Disconnected current lens is full length shot or short camera lens；

S2.4, extracts key frame respectively to long and short camera lens, extracts a key frame at random for short camera lens, and full length shot is then Multiple image is extracted as key frame by method at equal intervals；

S2.5, recombinates the keyframe sequence of extraction, reverts to generate on different visual angles direction video summary, observes Person by reaching the purpose of video fast browsing to the operation of video summary.

In S2.2, selecting absolute brightness frame difference AIFD as the characteristic quantity of measurement change degree of video content, it defines Formula is as follows:

$AIFD\left(t\right)=\underset{x=1}{\overset{W}{\mathrm{\Σ}}}\underset{y=1}{\overset{H}{\mathrm{\Σ}}}abs\left(f\right(x,y,t+1)-f(x,y,t\left)\right)---\left(4\right)$

In above formula and represent respectively under brightness value and the t of t picture frame pixel at coordinate in video sequence One frame is the brightness value of pixel at coordinate, W and H represents width and the height of frame of video respectively；If regarding in a certain view directions The complete picture frame number play of frequency is N, then the luminance frame difference average of video is:

$\stackrel{\‾}{AIFD\left(t\right)}=\frac{1}{N}AIFD\left(t\right)---\left(5\right)$

By calculating luminance frame difference average as determinating reference, two different coefficient a and b are set (when the value of a and b sets Time the least, easy flase drop；When setting the biggest, easy missing inspection.In experiment, the value of a is 1.2, and b value is 2.3, empirical value) to bright Whether degree frame difference average is weighted obtaining high-low threshold value thresh_low and thresh_high, change as camera lens and with which The decision condition that the mode of kind is changed, wherein

$\left\{\begin{array}{c}thresh\_low=a\·\stackrel{\‾}{AIFD\left(t\right)}\\ thresh\_high=b\·\stackrel{\‾}{AIFD\left(t\right)}\end{array}\right.---\left(6\right)$

In S2.2, the method splitting video sequence group is as follows:

First the video requency frame data of input is initialized, calculate the AIFD eigenvalue of adjacent two frames of t, compare also Judge the eigenvalue of present frame and the size of decision threshold, detect with this and whether there is camera lens between present frame and next frame and turn Change, it is determined that if method for present frame eigenvalue less than thresh_low; there is not Shot change, if more than thresh_low and Then think that less than thresh_high present frame there occurs that gradual shot is changed, if more than thresh_high, thinking that present frame is sent out Give birth to abrupt shot conversion, gradual shot conversion and abrupt shot conversion and all represented that present frame is recorded as there occurs turning of camera lens Changing, present frame is record camera lens switching node.

In S2.3, by the calculating to the quantity of motion summation calculating camera lens, and weigh door with the quantity of motion pre-set Limit compares, it is judged that camera lens belongs to full length shot or short camera lens, wherein represents the relative motion between adjacent two frame of video of t Amount, represents the persistent period of camera lens, is judged to full length shot when camera motion amount summation weighs thresholding more than quantity of motion, is otherwise Short camera lens.

72 degree of panoramic video fast browsing methods of one that the present invention proposes can be quickly generated virtual scene in difference See-through view on direction of visual lines, and on all directions view, analogue camera rotates and zoom motion effectively, improves virtual The surfing of scene, it is possible to well meet this specific application area of virtual reality system.

Accompanying drawing explanation

Fig. 1 panoramic picture back projection coordinate system schematic diagram

Segmentation restructuring block diagram on Fig. 2 panoramic video different views direction

Fig. 3 key-frame extraction block diagram

Fig. 4 is the trigonometric function relation schematic diagram between W, f, θ three

Detailed description of the invention

Below with reference to accompanying drawing, the present invention is described in further details.

In order to enable to carry out 720 degree of panoramic videos effectively browsing at a high speed on all directions view, the first of the present invention Step is to utilize Inverse Projection to be reconstructed panoramic video, obtains corresponding regarding on each direction of visual lines of sphere view space Graphic sequence, and the rotation of analogue camera and zoom motion realize browsing video in different visual angles, specifically comprises the following steps that

S1.1, completes the splicing of 720 degree of panoramic pictures, and sets up centered by the centre of sphere based on sphere view space model Two coordinate systems, represent world coordinate system XYZ and camera coordinate system xyz respectively.

Wherein, the method for the splicing completing 720 degree of panoramic pictures based on sphere view space model is: sit according to photographing unit Straight line vertical with image transverse axis in the image that straight line parallel with y-axis in mark system xyz generates according to sphere parameters transform is still For this character of vertical line, multiple real scene images of fish eye lens shooting are carried out rotation transformation correction, obtains every real scene image The azimuth information at view space of upper pixel, utilizes azimuth information to splice multiple images, eliminate real scene image it Between duplicate message that may be present, finally project on a sphere, store with the form of Spherical Panorama Image.

Wherein camera coordinate system xyz is by the world coordinate system XYZ X-axis anglec of rotation in world coordinate system, further around generation The Y-axis anglec of rotation in boundary's coordinate system and obtain.

Setting image S and splice complete Spherical Panorama Image as one, Q is any one pixel on Spherical Panorama Image S Point, image coordinate is；J is the view (i.e. J is a view on some direction of visual lines to be obtained) needing to generate, As it is shown in figure 1, some P is the point corresponding on view J for some Q on sphere, its image coordinate is；F represents pixel focal length, according to Shooting real scene image uses common lens (general wide-angle lens and standard lens) or fish eyes camera lens to estimate.

S1.2, in order to by unified for the basic unit of pixel under two coordinate systems, carry out the pixel focal distance f of camera lens Estimate.

The pixel focal length method of estimation of common lens (general wide-angle lens and standard lens) is: set camera level rotation The real scene image n circling captured opens, then the horizontal view angle of photographing unit is 360/n, and real scene image width is W, according to triangle Functional relationship can show that the pixel focal length estimation formulas of common lens is: f=W/ (2tan (180/n)).Wherein trigonometric function Relation refers to the relations such as the sine of the interior existence of right angled triangle, cosine, tangent.

With reference to Fig. 4, take a sectional drawing of panorama sketch, by trigonometric function relation, each amount can be indicated.Assume Figure represents with θ the horizontal view angle of photographing unit, then has θ=360/n, as seen from the figure the trigonometric function relation between W, f, θ three Can obtain:

Transformation resolution goes out, f=W/ (2tan (θ/2)), i.e. f=W/ (2tan (180/n))

Fish-eye pixel focal length is estimated to be derived by fish-eye equidistant imaging model, particularly as follows: will After the dark border of fish eye images is removed, the width of image is designated as W, and the most fish-eye pixel focal length estimation formulas is: f= W/ φ, wherein φ is fish-eye horizontal view, can be made a look up by fish eye lens description.

S1.3, generates the inverse operation of process, is under three-dimensional parameter coordinate by two dimensional image coordinate transformation according to panoramic picture Carry out computing, point on image coordinate point Q correspondence sphere, meet following transforming relationship formula:

$\left\{\begin{array}{c}u=fcos(\frac{\mathrm{\π}}{2}-\frac{y}{f})cos\left(\frac{x}{f}\right)\\ v=fsin(\frac{\mathrm{\π}}{2}-\frac{y}{f})\\ w=f\cos (\frac{\mathrm{\π}}{2}-\frac{y}{f})sin\left(\frac{x}{f}\right)\end{array}\right.---\left(1\right)$

Calculating the transformation matrix H of corresponding point under Two coordinate system, expression formula is as follows:

$H=\left[\begin{array}{ccc}1& 0& 0\\ 0& cos\mathrm{\α}& sin\mathrm{\α}\\ 0& -sin\mathrm{\α}& cos\mathrm{\α}\end{array}\right]\left[\begin{array}{ccc}cos\mathrm{\β}& 0& -\sin \mathrm{\β}\\ 0& 1& 0\\ sin\mathrm{\β}& 0& cos\mathrm{\β}\end{array}\right]---\left(2\right)$

After trying to achieve transformation matrix, from both the above formula, the point under coordinate system XYZ, correspondence under coordinate system xyz Coordinate be.

S1.4, it is known that in video, the width of captured real scene image frame is W, and height is H, sets up Spherical Panorama Image and takes up an official post The functional relation of some P corresponding on view J with it for 1 Q that anticipates, and calculate the coordinate of each corresponding point.

$\left\{\begin{array}{c}x=\frac{W}{2}-\frac{f\cos (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\cos (\frac{x^{\′}}{f}+\mathrm{\β})}{\cos (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\cos \mathrm{\α}\sin (\frac{x^{\′}}{f}+\mathrm{\β})-\sin (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\sin \mathrm{\α}}\\ y=\frac{H}{2}-\frac{f\sin (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\cos \mathrm{\α}+f\cos (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\sin \mathrm{\α}\sin (\frac{x^{\′}}{f}+\mathrm{\β})}{\cos (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\cos \mathrm{\α}\sin (\frac{x^{\′}}{f}+\mathrm{\β})-\sin (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\sin \mathrm{\α}}\end{array}\right.---\left(3\right)$

So far the first step of the present invention is completed, by the picture frame of 720 degree of panoramic videos through back project, depending on 720 degree of panoramic videos i.e. can be carried out seeing on any sight direction by view corresponding on space of points any sight direction See.Owing to directly 720 degree panoramic videos to be carried out comprehensive browsing one by one, there is the problem that access data volume is excessive that browses, one Aspect easily causes the fatigue of observer, on the other hand affect the efficiency of key message extraction, based on this, present invention additionally comprises the Two steps, second step extracts key frame according to the length shooting video lens under different scenes, is built by the key frame extracted One section of video frequency abstract, observer, by operating the keyframe sequence video of restructuring, reaches quick to 720 degree of panoramic videos The purpose browsed.

Video is made up of multiple different scenes, comprises multiple camera lens, wherein have full length shot, short mirror under each scene Head, each camera lens is formed according to certain played in order by multiframe associated images, and therefore frame of video is to form video base This unit.In order to be able to realize the fast browsing of video, obtaining the key frame in video image becomes the extraction of video effective information Key.It is said that in general, different types of video council according to self theme photographed scene had emphasis and time emphasis point, camera lens Length also can distinguish according to the difference of focus, therefore by detection and the length camera lens that judges in video, for pass The extraction of key frame is more favourable.

S2.1, the first step obtains panoramic video view sequence in different directions, by these video sequences according to projection View sequence on different directions visual angle carries out classifying, and (360 degree of panorama sketch that a width launches are by the image of multiple different visual angles Being spliced, be exactly that panorama sketch reverts to multiple views in different visual angles through back projection, these views are arranged according to sequence Put, classify according to sequence number), obtain the view sequence group that can individually browse on multiple visual angle.A frame panorama sketch in panoramic video What back projection obtained is the view on multiple direction of visual lines, and panoramic video back projection obtains is regarding in multiple view directions Frequently view sequence, each direction has multiple views.

S2.2, carries out dividing processing to the video sequence group on different directions visual angle respectively.

Select absolute brightness frame difference AIFD (absolute intensity frame difference) as weighing video The characteristic quantity of content change degree, its definition is as follows:

$AIFD\left(t\right)=\underset{x=1}{\overset{W}{\mathrm{\Σ}}}\underset{y=1}{\overset{H}{\mathrm{\Σ}}}abs\left(f\right(x,y,t+1)-f(x,y,t\left)\right)---\left(4\right)$

In above formula and represent brightness value and the t+1 moment of t picture frame pixel at coordinate in video sequence respectively Picture frame is the brightness value of pixel at coordinate, W and H represents width and the height of frame of video respectively.If in a certain view directions The picture frame number that video is completely play is N, then the luminance frame difference average of video is:

$\stackrel{\‾}{AIFD\left(t\right)}=\frac{1}{N}AIFD\left(t\right)---\left(5\right)$

Owing to the luminance frame difference change of pixel is little under same camera lens, present the more uniform distribution situation of ratio, therefore Two different coefficient a and b can be set by calculating luminance frame difference average as determinating reference, when the value of a and b sets the least Time, easy flase drop；When setting the biggest, easy missing inspection.(in experiment, the value of a is 1.2, and b value is 2.3, empirical value), to brightness Whether frame difference average is weighted obtaining high-low threshold value thresh_low and thresh_high, change as camera lens and with which kind of The decision condition that mode is changed.

$\left\{\begin{array}{c}thresh\_low=a\·\stackrel{\‾}{AIFD\left(t\right)}\\ thresh\_high=b\·\stackrel{\‾}{AIFD\left(t\right)}\end{array}\right.---\left(6\right)$

The step that implements splitting video sequence is first to initialize the video requency frame data of input, meter Calculate the AIFD eigenvalue of adjacent two frames of t, relatively and judge the eigenvalue of present frame and the size of decision threshold, examine with this Survey and between present frame and next frame, whether there is camera lens conversion.If decision method is present frame eigenvalue is less than thresh_low, There is not Shot change, if more than thresh_low and less than thresh_high, thinking that present frame may have occurred gradual change mirror Head conversion, if more than thresh_high, thinks that present frame may have occurred abrupt shot conversion, and either gradual change or sudden change are all Present frame is recorded as there occurs the conversion of camera lens, i.e. record camera lens switching node.

S2.3, component motion is commonly used to characterize the situation of content change in video, by the quantity of motion calculating camera lens The calculating of summation, and (it is threshold value set in advance that quantity of motion weighs thresholding, it is generally recognized that same to weigh thresholding with the quantity of motion arranged Under one camera lens, the histogram difference of two two field pictures is less, when difference aggregate-value i.e. quantity of motion summation exceedes the quantity of motion measurement of setting During thresholding, it is determined that for full length shot) compare, it is judged that camera lens belongs to full length shot or short camera lens, wherein represents adjacent two videos of t Amount of relative motion between frame, it is simply that the difference between adjacent two frames, weighs by histogram difference rate, even if same mirror Two two field pictures under Tou are the most constant, and simply difference value is less.Represent the persistent period of camera lens, when camera motion amount summation It is judged to full length shot when weighing thresholding more than quantity of motion, is otherwise short camera lens.

S2.4, extracts a key frame for short camera lens according to randomly selecting method, and full length shot is then according to camera lens start frame Start, equally spaced choose the multiple image key frame as full length shot.

S2.5, recombinates to the keyframe sequence extracted, and reverts to generate on different visual angles direction video summary, observes Person by reaching the purpose of video fast browsing to the operation of video summary.

Claims (10)

1. 720 degree of panoramic video fast browsing methods, it is characterised in that comprise the following steps:

720 degree of full-view video images are reconstructed by S1 first with Inverse Projection, obtain each sight line of sphere view space View sequence corresponding on direction；

S2, judges camera lens length by calculating the absolute brightness frame difference of adjacent image frame in video sequence, then extracts key Frame, it is achieved panoramic video fast browsing.

2. according to 720 described in claim 1 degree panoramic video fast browsing method, it is characterised in that the method for S1 is as follows:

S1.1, completes the splicing of 720 degree of panoramic pictures, and sets up two centered by the centre of sphere based on sphere view space model Coordinate system, represents world coordinate system XYZ and camera coordinate system xyz respectively；Wherein camera coordinate system xyz is by world coordinates It is XYZ X-axis rotation alpha angle in world coordinate system, obtains further around the Y-axis rotation β angle in world coordinate system；

S1.2, by unified for the basic measuring unit of pixel under Two coordinate system in S1.1, then to calculate using pixel as fundamental quantity Each pixel is i.e. estimated the viewpoint pixel focal distance f to view plane under camera coordinate system by the pixel focal length of degree unit；

S1.3, sets up the coordinate of X-Y scheme picture point and the conversion pass of the three-dimensional parameter coordinate points corresponding to sphere by pixel focal distance f It is formula, further according to world coordinate system XYZ X-axis rotation alpha angle in world coordinate system, revolves further around the Y-axis in world coordinate system During turning β angle, along with the rotation of coordinate axes, pixel expression on each coordinate components also occurs to change accordingly, This change can be closed with trigonometric function and be tied up to show on respective coordinate components, thus obtains corresponding point under Two coordinate system Transformation matrix H；

S1.4, is set up counter-transformation function by transformation matrix H, find out arbitrfary point on panoramic picture to spherical space each view point Corresponding relation formula, calculates each point coordinates, obtains view corresponding on each direction of visual lines of view space.

3. according to 720 described in claim 1 degree panoramic video fast browsing method, it is characterised in that in S1.2, if image S is a complete Spherical Panorama Image of splicing, and Q is any one pixel on Spherical Panorama Image S, image coordinate be (x ', y′)；J is the view needing to generate, and some P is the point corresponding on view J for some Q on Spherical Panorama Image, its image coordinate For (x, y)；F represents pixel focal length, and f uses camera lens to estimate according to shooting real scene image；

The method of estimation of the pixel focal distance f of wide-angle lens or standard lens is: sets camera level and rotates a circle captured reality Scape image n opens, then the horizontal view angle of photographing unit is 360/n, and real scene image width is W, can draw according to trigonometric function relation The pixel focal length estimation formulas of common lens is: f=W/ (2tan (180/n))；

The method of estimation of fish-eye pixel focal distance f is: the width note of image after the dark border of fish eye images being removed For W, the most fish-eye pixel focal length estimation formulas is: f=W/ φ, wherein φ is fish-eye horizontal view.

4. according to 720 described in claim 3 degree panoramic video fast browsing method, it is characterised in that in S1.3, use pixel Focal distance f sets up the transforming relationship formula of the coordinate of X-Y scheme picture point and the three-dimensional parameter coordinate points corresponding to sphere, as follows:

$\left\{\begin{array}{c}u=fcos(\frac{\mathrm{\π}}{2}-\frac{y}{f})cos\left(\frac{x}{f}\right)\\ v=fsin(\frac{\mathrm{\π}}{2}-\frac{y}{f})\\ w=fcos(\frac{\mathrm{\π}}{2}-\frac{y}{f})sin\left(\frac{x}{f}\right)\end{array}\right.---\left(1\right)$

Calculating the transformation matrix H of corresponding point under Two coordinate system, expression formula is as follows:

$H=\left[\begin{array}{ccc}1& 0& 0\\ 0& cos\mathrm{\α}& sin\mathrm{\α}\\ 0& -sin\mathrm{\α}& cos\mathrm{\α}\end{array}\right]\left[\begin{array}{ccc}cos\mathrm{\β}& 0& -\sin \mathrm{\β}\\ 0& 1& 0\\ sin\mathrm{\β}& 0& cos\mathrm{\β}\end{array}\right]---\left(2\right).$

720 degree of panoramic video fast browsing methods the most according to claim 4, it is characterised in that in S1.4, by S1.3 Formula (1) and (2) understand, (u, v, w), coordinate corresponding under coordinate system xyz is (u ', v ', w ') for some Q ' under coordinate system XYZ =H (u, v, w)；

In known video, the width of captured real scene image is W, and height is H, set up any point Q on Spherical Panorama Image (x ', Y ') the some P corresponding on view J with some Q (x, functional relation y), and utilize formula (3) to calculate the coordinate of each corresponding point, Obtain view corresponding on each direction of visual lines of view space；

$\left\{\begin{array}{c}x=\frac{W}{2}-\frac{f\cos (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\cos (\frac{x^{\′}}{f}+\mathrm{\β})}{\cos (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\cos \mathrm{\α}\sin (\frac{x^{\′}}{f}+\mathrm{\β})-\sin (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\sin \mathrm{\α}}\\ y=\frac{H}{2}-\frac{f\sin (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\cos \mathrm{\α}+f\cos (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\sin \mathrm{\α}\sin (\frac{x^{\′}}{f}+\mathrm{\β})}{\cos (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\cos \mathrm{\α}\sin (\frac{x^{\′}}{f}+\mathrm{\β})-\sin (\frac{\mathrm{\π}}{2}-\frac{y^{\′}}{f})\sin \mathrm{\α}}\end{array}\right.---\left(3\right).$

720 degree of panoramic video fast browsing methods the most according to claim 1, it is characterised in that the method for S2 is as follows:

S2.1, carries out structuring process to panoramic video sequences, and video sequence step S1 obtained is according to being projected in not Tongfang View frames sequence on visual angle is classified, and obtains the video sequence group that can individually browse on multiple visual angle；

S2.2, splits the video sequence group of each view directions respectively, and in calculating video sequence, adjacent image frame is absolute Luminance frame is poor, it is judged that the switching node of video lens, is multiple camera lens fragments by video signal process；

S2.3, calculates the quantity of motion summation of each camera lens fragment, sets quantity of motion and weighs thresholding, judges to work as according to shot durations Front camera lens is full length shot or short camera lens；

S2.4, extracts key frame respectively to long and short camera lens, extracts a key frame at random for short camera lens, and full length shot is then pressed Interval method extracts multiple image as key frame；

S2.5, recombinates the keyframe sequence of extraction, reverts to generate on different visual angles direction video summary, and observer is led to Cross the operation to video summary and reach the purpose of video fast browsing.

720 degree of panoramic video fast browsing methods the most according to claim 6, it is characterised in that in S2.2, select definitely Luminance frame difference AIFD is as the characteristic quantity of measurement change degree of video content, and its definition is as follows:

$AIFD\left(t\right)=\underset{x=1}{\overset{W}{\mathrm{\Σ}}}\underset{y=1}{\overset{H}{\mathrm{\Σ}}}abs\left(f\right(x,y,t+1)-f(x,y,t\left)\right)---\left(4\right)$

In above formula, (with f (x, y, t+1), x, y t) represent that in video sequence, t picture frame is in (x, y) pixel at coordinate to f respectively Point brightness value and t next frame (x, y) brightness value of pixel at coordinate, W and H represent respectively frame of video width and Highly；If the picture frame number that in a certain view directions, video is completely play is N, then the luminance frame difference average of video is:

$\stackrel{\‾}{AIFD\left(t\right)}=\frac{1}{N}AIFD\left(t\right)---\left(5\right)$

By calculating luminance frame difference average as determinating reference, two different coefficient a and b are set, luminance frame difference average is carried out Weighting obtains high-low threshold value thresh_low and thresh_high, and whether that changes as camera lens and change in which way sentences Fixed condition, wherein

$\left\{\begin{array}{c}thresh\_low=a\·\stackrel{\‾}{AIFD\left(t\right)}\\ thresh\_high=b\·\stackrel{\‾}{AIFD\left(t\right)}\end{array}\right.---\left(6\right).$

720 degree of panoramic video fast browsing methods the most according to claim 7, it is characterised in that the value of a is 1.2, b Value is 2.3.

720 degree of panoramic video fast browsing methods the most according to claim 7, it is characterised in that in S2.2, to video sequence The method that row group carries out splitting is as follows:

First the video requency frame data of input is initialized, calculate the AIFD eigenvalue of adjacent two frames of t, relatively and judge The eigenvalue of present frame and the size of decision threshold, detect with this and whether there is camera lens conversion between present frame and next frame, If less than thresh_low, there is not Shot change, if being more than thresh_low and being less than for present frame eigenvalue in decision method Thresh_high then thinks that present frame there occurs that gradual shot is changed, if more than thresh_high, thinking that present frame there occurs Abrupt shot is changed, and gradual shot conversion and abrupt shot conversion all represent that present frame is recorded as there occurs the conversion of camera lens, when Front frame is record camera lens switching node.

720 degree of panoramic video fast browsing methods the most according to claim 9, it is characterised in that in S2.3, by right Calculate the quantity of motion summation of camera lensCalculating, and weigh thresholding with the quantity of motion that pre-sets and compare, it is judged that Camera lens belongs to full length shot or short camera lens, and wherein M_f (t) represents the amount of relative motion between adjacent two frame of video of t, S_ Time represents the persistent period of camera lens, is judged to full length shot, otherwise when camera motion amount summation weighs thresholding more than quantity of motion For short camera lens.