CN106372598A - Image stabilizing method based on image characteristic detection for eliminating video rotation and jittering - Google Patents

Abstract

The invention provides an image stabilizing method based on image characteristic detection for eliminating video rotation and jittering. The method comprises steps that step S1, a video adjacent frame is acquired; step S2, characteristic points are extracted, and characteristic description vectors are generated; step S3, characteristic point matching is carried out, step S4, characteristic point pairs in error matching are eliminated; step S5, a rotation angle is calculated; step S6, reverse superposition of a rotation angle of a second video image is carried out, and video rotation is eliminated; step S7, characteristic point locus for the video frame image after video rotation elimination is generated; step S8, Gauss noise of the characteristic point locus is eliminated; step S9, an accurate video jittering direction and amplitude are acquired; and step S10, jittering elimination operation on the video frame image is carried out. Through the method, jittering and rotation of the image can be eliminated, algorithm optimization is carried out for extreme point detection, characteristic principal direction generation and description vector construction, an operation speed is greatly improved, and error matching of adjacent frame characteristic points can be effectively avoided through utilizing a nearest neighbor to next nearest neighbor method and an RANSIC algorithm.

Description

A kind of digital image stabilization method eliminating video rotation and shake based on Image Feature Detection

Technical field

The present invention relates to surely as technical field and in particular to a kind of elimination video based on Image Feature Detection rotates and trembles Dynamic digital image stabilization method.

Background technology

In recent years, with the popularization of photographic equipment, video is increasingly appearing in the work and life of people, for example Safety monitoring, automobile monitor, digital camera etc..When photographic equipment is operated on the motion carrier such as vehicle-mounted, hand-held, equipment meeting Occur corresponding move, cause the shake of video, rotation simultaneously, scale.For individual, the shake of video, Effect of Rotation video Viewing effect, for automatic identification tracing system, then lead to system detectio mistake or follow the trail of unsuccessfully.Existing digital image stabilization method The eradicating efficacy of the rotation to video and shake is not fine.

In view of drawbacks described above, creator of the present invention passes through long research and practice obtains the present invention finally.

Content of the invention

For solving above-mentioned technological deficiency, the technical solution used in the present invention is, provides one kind to be based on Image Feature Detection The digital image stabilization method eliminating video rotation and shake, the method comprises the following steps:

Step s1, obtains video consecutive frame；

Step s2, extracts the characteristic point of the adjacent two field picture of video using surf algorithm and generates feature description vector；

Step s3, the characteristic point of traversal video frame images and the distance of the characteristic point of adjacent two field picture, using arest neighbors ratio Secondary near neighbor method matching characteristic point；

Step s4, rejects the paired characteristic point of error hiding using ransic algorithm；

Step s5, calculates the anglec of rotation of described adjacent video two field picture；

Step s6, is reversely superimposed the anglec of rotation to the second frame video image, eliminates video rotation；

Step s7, to eliminating the postrotational video frame images of video, generates characteristic point rail using klt feature point tracking algorithm Mark；

Step s8, carries out kalman Filtering Processing to described feature point trajectory, and the Gauss eliminating described feature point trajectory makes an uproar Sound；

Step s9, carries out b spline curve fitting to feature point trajectory after filtering, obtains accurate video jitter direction and width Value；

Video frame images, according to described video jitter direction and amplitude, are carried out eliminating dither operation by step s10.

Preferably, described step s2 specifically includes following steps:

Step s21, does convolution using box filtering and image, builds multiscale space, generate metric space pyramid；

Step s22, asks for the local extremum under a certain particular dimensions, carries out non-maximum in the three-dimensional field of its 3*3*3 Suppression, screening qualified point is candidate's extreme value, record position and size；

Step s23, in metric space and image space to extreme value near click-through row interpolation, filter the pole of low contrast Value point；

Step s24, constructs a principal direction to each characteristic point so that surf feature keeps constant to image rotation；

Step s25, generates feature description vector to each characteristic point, is normalized operation so that feature point pairs brightness Change, view transformation keep constant.

Preferably, in described step s24, a principal direction is constructed to each characteristic point comprising the following steps:

Step s241, marking the center of circle is sample characteristics point position, and radius is the border circular areas of certain setting value；

Step s242, calculates the harr small echo response vector in its x, y direction for all characteristic points in border circular areas respectively；

Step s243, gives different Gauss weights to each direction vector, near the center of circle, imparting that direction contribution is big Greater weight；

Step s244, divides 6 sectors by circular, calculates in each sector region, the harr small echo response vector of weighting Vector, with maximum vector and corresponding direction for the principal direction of this feature point.

Preferably, characteristic point generates feature description vector and specifically includes following steps in described step s25:

Coordinate axess are rotated to the principal direction of this feature point by step s251 first, determine with characteristic point as midpoint, the length of side is The square field of 20 σ；

Step s252, principal direction region division is become 4 × 4 sub-block regions, and every sub-regions sample according to 5 × 5 sizes, Harr wavelet filtering process, wherein d are carried out to each block region_xRepresent harr small echo level of response component, d_yRepresent that harr is little Ripple responds vertical component；

Step s253, needs to carry out Gaussian function weighting operations to the response of harr small echo, to strengthen the Shandong to geometric transformation Rod, the center of this Gaussian function is characteristic point, and σ₀=3.3 σ；

Step s254, calculates the small echo response value with vertical direction in the horizontal direction of 25 sampled points of 16 sub-regions And its summation of absolute value, thus obtain a new vector: ∑ d_x, ∑ d_y, ∑ | d_x|, ∑ | d_y|；

Step s255, is normalized operation to the vector of 16 × 4=64 dimension of each characteristic point, removes brightness flop and make The impact becoming, thus obtain the description vectors of characteristic point.

Preferably, arest neighbors in described step s3 specifically includes following steps than secondary near neighbor method matching characteristic point:

Step s31, to the sample characteristics point in image, calculates and its closest and secondary near similitude, order is recently Distance is d₁, secondary is closely d₂, using Euclidean distance formula, its expression formula is as follows for wherein range formula:

$d=\sqrt{\underset{i=1}{\overset{n}{\mathrm{\σ}}}{(x_{1i}-x_{2i})}^2}$

Step s32, calculates minimum distance and time in-plant ratio, ratio is compared with given threshold:

When ratio is less than threshold value it is believed that this feature Point matching, otherwise it is assumed that mismatching, give up this feature point.

Preferably, described step s4 specifically includes following steps:

Step s41, randomly drawing sample set p from data acquisition system s；

Step s42, according to sample set p solving model parameter, thus obtain system model h；

Step s43, to remaining data acquisition systemElement l, the element l in statistics set l be System the distance between model h, if distance is less than given threshold, for interior point, otherwise for exterior point；

Step s44, repeats described step s41, step s42, step s43, by most for interior quantity once right System model h answered, as final mask, filters the exterior point under this model.

Preferably, described step s7 specifically includes following steps:

Step s71, selects video, with the first frame as reference frame, extracts characteristic point and preserves, recording feature point number m；

Step s72, in the next frame, using klt track algorithm track and extract characteristic point and preserve, recording feature point number n；

Step s73, judges the threshold value whether ratio of n and m sets less than certain, if less than threshold value, then generates one section Feature point trajectory, again with present frame as reference frame, returns to described step s71；

Step s74, judges whether this video terminates, if do not terminated, returns to step s72.

Prior art compare the beneficial effects of the present invention is: the present invention provide a kind of disappearing based on Image Feature Detection Except the digital image stabilization method of video rotation and shake, enable to eliminate flating and rotation based on the surf algorithm of scale invariant feature Turn；Surf algorithm uses for reference the design philosophy of sift algorithm, keeps the detection performance that it is good, in extreme point detection, feature main formula Do algorithm optimization to when generating and description vectors build, substantially increase calculating speed；Using arest neighbors than secondary near neighbor method The error hiding of consecutive frame characteristic point can be efficiently avoid with ransic algorithm.

Brief description

For the technical scheme being illustrated more clearly that in various embodiments of the present invention, below will be to required in embodiment description The accompanying drawing using is briefly described.

Fig. 1 is a kind of flow process of digital image stabilization method eliminating video rotation and shake based on Image Feature Detection of the present invention Figure；

Fig. 2 is the flow chart of step s2；

Fig. 3 is the flow chart constructing a principal direction to each characteristic point in step s24；

Fig. 4 is the flow chart that in step s25, characteristic point generates feature description vector.

Specific embodiment

Below in conjunction with accompanying drawing, the above-mentioned He other technical characteristic of the present invention and advantage are described in more detail.

As shown in figure 1, a kind of image stabilization eliminating video rotation and shake based on Image Feature Detection for the present invention The flow chart of method, the method comprises the following steps:

Step s1, obtains video consecutive frame.

Step s2, extracts the characteristic point of the adjacent two field picture of video using surf algorithm and generates feature description vector.

Step s3, the characteristic point of traversal video frame images and the distance of consecutive frame image characteristic point, using arest neighbors than secondary Near neighbor method matching characteristic point.

Step s4, rejects the paired characteristic point of error hiding using ransic algorithm.

Step s5, calculates the anglec of rotation of adjacent video two field picture.

Step s6, is reversely superimposed the anglec of rotation to the second frame video image, eliminates video rotation.

Step s7, to eliminating the postrotational video frame images of video, generates characteristic point rail using klt feature point tracking algorithm Mark.

Step s8, carries out kalman Filtering Processing to feature point trajectory, eliminates the Gaussian noise of feature point trajectory.

Step s9, carries out b spline curve fitting to feature point trajectory after filtering, obtains accurate video jitter direction and width Value.

Video frame images, according to video jitter direction and amplitude, are carried out eliminating dither operation by step s10.

As shown in Fig. 2 for the flow chart of step s2, specifically including following steps:

Step s21, does convolution using box filtering and image, builds multiscale space, generate metric space pyramid.

Step s22, asks for the local extremum under a certain particular dimensions, carries out non-maximum in the three-dimensional field of its 3*3*3 Suppression, screening qualified point is candidate's extreme value, record position and size.

Step s23, in metric space and image space to extreme value near click-through row interpolation, filter the pole of low contrast Value point.

Step s24, constructs a principal direction to each characteristic point so that surf feature keeps constant to image rotation.

Step s25, generates feature description vector to each characteristic point, is normalized operation so that feature point pairs brightness Change, view transformation keep constant.

As shown in figure 3, the flow chart for a principal direction being constructed to each characteristic point in step s24, specifically include following Step:

Step s241, marking the center of circle is sample characteristics point position, and radius is the border circular areas of certain setting value.

Step s242, calculates the harr small echo response vector in its x, y direction for all characteristic points in border circular areas respectively.

Step s243, gives different Gauss weights to each direction vector, near the center of circle, imparting that direction contribution is big Greater weight.

Step s244, divides 6 sectors by circular, calculates in each sector region, the harr small echo response vector of weighting Vector, with maximum vector and corresponding direction for the principal direction of this feature point.

As shown in figure 4, the flow chart generating feature description vector for characteristic point in step s25, specifically include following steps:

Coordinate axess are rotated to the principal direction of this feature point by step s251 first, determine with characteristic point as midpoint, the length of side is The square field of 20 σ (wherein σ is characterized a corresponding yardstick).

Step s252, principal direction region division is become 4 × 4 sub-block regions, and every sub-regions sample according to 5 × 5 sizes, Harr wavelet filtering process, wherein d are carried out to each block region_xRepresent harr small echo level of response component, d_yRepresent that harr is little Ripple responds vertical component.

Step s253, needs to carry out Gaussian function weighting operations to the response of harr small echo, to strengthen the Shandong to geometric transformation Rod, the center of this Gaussian function is characteristic point, and σ₀=3.3 σ.

Step s254, calculates the small echo response value with vertical direction in the horizontal direction of 25 sampled points of 16 sub-regions And its summation of absolute value, thus obtain a new vector: σ d_x, σ d_y, σ | d_x|, ∑ | d_y|.

Step s255, is normalized operation to the vector of 16 × 4=64 dimension of each characteristic point, removes brightness flop and make The impact becoming, thus obtain the description vectors of characteristic point.

Arest neighbors in step s3 specifically includes following steps than secondary near neighbor method matching characteristic point:

Step s31, to the sample characteristics point in image, calculates and its closest and secondary near similitude, order is recently Distance is d₁, secondary is closely d₂, using Euclidean distance formula, its expression formula is as follows for wherein range formula:

$d=\sqrt{\underset{i=1}{\overset{n}{\mathrm{\σ}}}{(x_{1i}-x_{2i})}^2}$

Step s32, calculates minimum distance and time in-plant ratio, ratio is compared with given threshold:

When ratio is less than threshold value it is believed that this feature Point matching, otherwise it is assumed that mismatching, give up this feature point.

Step s4 step can more accurately obtain the consecutive frame anglec of rotation.Specifically include following steps:

Step s41, randomly drawing sample set p from data acquisition system s.

Step s42, according to sample set p solving model parameter, thus obtain system model h；

Step s43, to remaining data acquisition systemElement l, the element l in statistics set l be System the distance between model h, if distance is less than given threshold, for interior point, otherwise for exterior point.

Step s44, repeated execution of steps s41, step s42, step s43, will be most for interior quantity once corresponding System model h, as final mask, filters the exterior point under this model.

Step s7 can improve the accuracy of feature point trajectory.Specifically include following steps:

Step s71, selects video, with the first frame as reference frame, extracts characteristic point and preserves, recording feature point number m.

Step s72, in the next frame, using klt track algorithm track and extract characteristic point and preserve, recording feature point number n.

Step s73, judges the threshold value whether ratio of n and m sets less than certain, if less than threshold value, then generates one section Feature point trajectory, again with present frame as reference frame, returns to step s71.

Step s74, judges whether this video terminates, if do not terminated, returns to step s72.

Step s8 specifically includes: in kalman Filtering Model, the time of day x in k moment_kTime of day by the k-1 moment Derive, expression formula is as follows:

x_k=f_kx_k-1+b_ku_k+w_k

Wherein f_kIt is to act on x_k-1On state transition matrix, b_kIt is that input controls matrix, w_kFor process noise, just meet State is distributed.In this application, the characteristic point position x in k moment_kIt is the characteristic point position x in k-1 moment_k-1Move position plus characteristic point Move d (x, y), wherein moving displacement can be by the product representation of the translational speed of image and inter frame temporal it is contemplated that inter frame temporal Shorter, the translational speed of image approximates the translational speed in a moment.

In k moment, time of day x_kWith observer state z_kMeet expression formula:

z_k=h_kx_x+v_k

Wherein h_kIt is observer state matrix, v_kFor observation noise, meet normal distribution.In the application, measurement is subject to only noise Interference, therefore observer state matrix h_kFor unit matrix.

In kalman filtering, kalman gain is constantly revised according to least mean-square error, using to current shape The predictive value that the observation optimization of state obtains in forecast period, to obtain more accurately estimated value.

A kind of digital image stabilization method eliminating video rotation and shake based on Image Feature Detection that the present invention provides is logical first Cross the characteristic point that Image Feature Detection Algorithms find video image, reject the characteristic point of wherein misrecognition；To adjacent video frames figure The characteristic point of picture is mated, and calculates the angle of adjacent video frames rotation, eliminates video rotation by angle compensation；Then right One section of video eliminating rotation, generates the feature point trajectory of this two field picture by feature point tracking algorithm, using filtering and curve Approximating method obtains the motion model of each two field picture, eliminates the shake of video by translation compensation.Have the advantage that

1. commonly use electronic image stabilization method, for example soon coupling, Gray Projection method, the motion model of translation can only be processed, that is, Flating can only be eliminated, and enable to eliminate flating and rotation based on the surf algorithm of scale invariant feature.

2. traditional sift feature detection algorithm has invariance to rotation, scaling, and to noise, light differential There is good robustness, but calculating speed is slow.Surf algorithm uses for reference the design philosophy of sift algorithm, keeps the detection that it is good Performance, has done algorithm optimization in extreme point detection, the generation of feature principal direction and description vectors when building, has substantially increased calculating speed Degree.

3. the mistake of consecutive frame characteristic point can be efficiently avoid than secondary near neighbor method and ransic algorithm using arest neighbors Coupling.Wherein arest neighbors solves consecutive frame than secondary near neighbor method and does not have similar features point or have asking of multiple similar features point Topic, ransic algorithm rejects Mismatching point according to the consistent degree with model.

The foregoing is only presently preferred embodiments of the present invention, be merely illustrative for the purpose of the present invention, and non-limiting 's.Those skilled in the art understands, it can be carried out in the spirit and scope that the claims in the present invention are limited with many changes, Modification, in addition equivalent, but fall within protection scope of the present invention.

Claims (7)

1. a kind of the digital image stabilization method of video rotation and shake eliminated it is characterised in that the method bag based on Image Feature Detection Include following steps:

Step s1, obtains video consecutive frame；

Step s2, extracts the characteristic point of the adjacent two field picture of video using surf algorithm and generates feature description vector；

Step s3, the characteristic point of traversal video frame images and the distance of the characteristic point of adjacent two field picture, nearer than secondary using arest neighbors Adjacent method matching characteristic point；

Step s4, rejects the paired characteristic point of error hiding using ransic algorithm；

Step s5, calculates the anglec of rotation of described adjacent video two field picture；

Step s6, is reversely superimposed the anglec of rotation to the second frame video image, eliminates video rotation；

Step s7, to eliminating the postrotational video frame images of video, generates feature point trajectory using klt feature point tracking algorithm；

Step s8, carries out kalman Filtering Processing to described feature point trajectory, eliminates the Gaussian noise of described feature point trajectory；

Step s9, carries out b spline curve fitting to feature point trajectory after filtering, obtains accurate video jitter direction and amplitude；

Video frame images, according to described video jitter direction and amplitude, are carried out eliminating dither operation by step s10.

2. a kind of digital image stabilization method eliminating video rotation and shake based on Image Feature Detection according to claim 1, It is characterized in that, described step s2 specifically includes following steps:

Step s21, does convolution using box filtering and image, builds multiscale space, generate metric space pyramid；

Step s22, asks for the local extremum under a certain particular dimensions, carries out non-maximum suppression in the three-dimensional field of its 3*3*3, Screening qualified point is candidate's extreme value, record position and size；

Step s23, in metric space and image space to extreme value near click-through row interpolation, filter the extreme value of low contrast Point；

Step s24, constructs a principal direction to each characteristic point so that surf feature keeps constant to image rotation；

Step s25, to each characteristic point generate feature description vector, be normalized operation so that feature point pairs brightness flop, View transformation keeps constant.

3. a kind of digital image stabilization method eliminating video rotation and shake based on Image Feature Detection according to claim 2, It is characterized in that, in described step s24, a principal direction is constructed to each characteristic point and comprises the following steps:

Step s241, marking the center of circle is sample characteristics point position, and radius is the border circular areas of certain setting value；

Step s242, calculates the harr small echo response vector in its x, y direction for all characteristic points in border circular areas respectively；

Step s243, gives different Gauss weights to each direction vector, near the center of circle, imparting that direction contribution is big larger Weight；

Step s244, divides 6 sectors by circular, calculates in each sector region, the arrow of the harr small echo response vector of weighting Amount and, with maximum vector and corresponding direction for the principal direction of this feature point.

4. a kind of digital image stabilization method eliminating video rotation and shake based on Image Feature Detection according to claim 3, It is characterized in that, in described step s25, characteristic point generates feature description vector and specifically includes following steps:

Coordinate axess are rotated to the principal direction of this feature point by step s251 first, determine with characteristic point as midpoint, the length of side is 20 σ Square field；

Step s252, principal direction region division is become 4 × 4 sub-block regions, every sub-regions sample according to 5 × 5 sizes, to every Individual block region carries out harr wavelet filtering process, wherein d_xRepresent harr small echo level of response component, d_yRepresent that harr small echo rings Answer vertical component；

Step s253, needs to carry out Gaussian function weighting operations to the response of harr small echo, to strengthen the robustness to geometric transformation, The center of this Gaussian function is characteristic point, and σ₀=3.3 σ；

Step s254, calculate 16 sub-regions 25 sampled points in the horizontal direction with the small echo response value of vertical direction and its The summation of absolute value, thus obtain a new vector: ∑ d_x, ∑ d_y, σ | d_x|, ∑ | d_y|；

Step s255, is normalized operation to the vector of 16 × 4=64 dimension of each characteristic point, removes what brightness flop caused Impact, thus obtain the description vectors of characteristic point.

5. a kind of digital image stabilization method eliminating video rotation and shake based on Image Feature Detection according to claim 4, It is characterized in that, the arest neighbors in described step s3 specifically includes following steps than secondary near neighbor method matching characteristic point:

Step s31, to the sample characteristics point in image, calculates and its closest and secondary near similitude, makes minimum distance For d₁, secondary is closely d₂, using Euclidean distance formula, its expression formula is as follows for wherein range formula:

$d=\sqrt{\underset{i=1}{\overset{n}{\σ}}{(x_{1i}-x_{2i})}^2}$

Step s32, calculates minimum distance and time in-plant ratio, ratio is compared with given threshold:

When ratio is less than threshold value it is believed that this feature Point matching, otherwise it is assumed that mismatching, give up this feature point.

6. a kind of digital image stabilization method eliminating video rotation and shake based on Image Feature Detection according to claim 5, It is characterized in that, described step s4 specifically includes following steps:

Step s41, randomly drawing sample set p from data acquisition system s；

Step s42, according to sample set p solving model parameter, thus obtain system model h；

Step s43, to remaining data acquisition systemElement l, the element l in statistics set l and system mould The distance between type h, if distance is less than given threshold, for interior point, otherwise for exterior point；

Step s44, repeats described step s41, step s42, step s43, will be most for interior quantity once corresponding System model h, as final mask, filters the exterior point under this model.

7. a kind of digital image stabilization method eliminating video rotation and shake based on Image Feature Detection according to claim 6, It is characterized in that, described step s7 specifically includes following steps:

Step s71, selects video, with the first frame as reference frame, extracts characteristic point and preserves, recording feature point number m；

Step s72, in the next frame, using klt track algorithm track and extract characteristic point and preserve, recording feature point number n；

Step s73, judges the threshold value whether ratio of n and m sets less than certain, if less than threshold value, then generates one section of feature The locus of points, again with present frame as reference frame, returns to described step s71；

Step s74, judges whether this video terminates, if do not terminated, returns to step s72.