CN107135331B - The UAV Video antihunt means and device of low-latitude flying scene - Google Patents
The UAV Video antihunt means and device of low-latitude flying scene Download PDFInfo
- Publication number
- CN107135331B CN107135331B CN201710198431.7A CN201710198431A CN107135331B CN 107135331 B CN107135331 B CN 107135331B CN 201710198431 A CN201710198431 A CN 201710198431A CN 107135331 B CN107135331 B CN 107135331B
- Authority
- CN
- China
- Prior art keywords
- characteristic point
- track
- coordinate
- point
- matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/144—Movement detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/681—Motion detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
- H04N23/682—Vibration or motion blur correction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/21—Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The present invention provides the UAV Video antihunt means and device of a kind of low-latitude flying scene, wherein method includes: to obtain UAV Video to be stablized, feature point extraction is carried out to each frame in video, connection obtains the track of each characteristic point, track is divided into long track and short track by threshold value, the corresponding global change's matrix of every frame picture can be calculated based on long track and smooth long track, in conjunction with the corresponding global change's matrix of every frame picture, short track and low-pass filter, available smooth short track, finally long track is combined to be calculated with short track using more plane optimizing methods, obtain stable UAV Video.The present invention is classified feature point trajectory, while guaranteeing characteristic point abundance regional stability effect, stablizing effect is played to the insufficient region of characteristic point, the influence that more plane optimizing methods edge is unstable in low-latitude flying scene can be solved to a certain extent, to improve the stablizing effect of UAV Video.
Description
Technical field
The present invention relates to field of communication technology more particularly to a kind of UAV Video antihunt means of low-latitude flying scene and
Device.
Background technique
UAV Video stabilization technique mainly causes to solve the reasons such as atmospheric turbulance, the vibration of rotation paddle and posture changing
UAV Video float problem.The technology passes through computer only on the basis of the image information in unmanned plane shooting video
Graph transformation restores stable field of view, is of great significance to enhancing unmanned plane shooting video usability.
It, will be a certain mainly based on estimating the transformation matrix between image sequence in current UAV Video antihunt means
Frame compensates all frames as benchmark, makes the visual angle of all frames close to selected frame, can also carry out to transformation matrix smoothly,
To obtain a series of metastable images.Due to such methods to every frame only with linear transformation, thus algorithm robustness
Well, processing speed is fast, is suitble to the use when high aerial and video content is stablized.However, in the photographed scene of low latitude, current nothing
Man-machine video stabilizing method still has a deficiency, on the one hand not can solve the problem of parallax experienced of scene in low latitude, on the other hand also vulnerable to
The influence of roller shutter effect.
Summary of the invention
The present invention provides the UAV Video antihunt means and device of a kind of low-latitude flying scene, for solve it is existing nobody
The problem of machine video stabilization technology low-to-medium altitude flying scene poor availability,.
The first aspect of the invention is to provide a kind of UAV Video antihunt means of low-latitude flying scene, comprising:
The characteristic point in the every frame picture of video to be stablized is extracted, the track for obtaining each characteristic point is connected and track is long
Degree；
The path length of characteristic point is compared with feature point trajectory length threshold, respectively obtain the long track of characteristic point and
The short track of characteristic point；
The coordinate value of the long track of characteristic point is smoothed using low-pass filtering, obtains the long rail of smoothed out characteristic point
Mark；
More plane optimizing calculating are carried out to the long track of characteristic point and the long track of smoothed out characteristic point, obtain every frame picture
Corresponding global change's matrix；
Transformation and smooth place are carried out using coordinate of global change's matrix to corresponding characteristic point in the short track of characteristic point
Reason, obtains the short track of smoothed out characteristic point；
The smoothed out long track of characteristic point and the short track of smoothed out characteristic point are merged to obtain characteristic point flat
Sliding rail mark calculates each feature point trajectory and characteristic point smooth track using more plane optimizing methods, obtains every frame figure
The corresponding object transformation matrix of piece；
The each pixel successively treated in the every frame picture of stable video carries out coordinate change using corresponding object transformation matrix
It changes, obtains stable UAV Video.
Further, described that more plane optimizing meters are carried out to the long track of characteristic point and the long track of smoothed out characteristic point
It calculates, obtains the corresponding global change's matrix of every frame picture, comprising:
For every frame picture, the coordinate of the corresponding each characteristic point of picture described in the long track of each characteristic point is obtained；
Obtain the coordinate of the corresponding each characteristic point of picture described in the long track of smoothed out characteristic point；
Using more plane optimizing methods to the coordinate of the corresponding each characteristic point of picture described in the long track of each characteristic point,
And the coordinate of the corresponding each characteristic point of picture described in the smoothed out long track of characteristic point is calculated, and the picture is obtained
Corresponding global change's matrix.
Further, described to be become using coordinate of global change's matrix to corresponding characteristic point in the short track of characteristic point
It changes and smoothing processing, obtains the short track of smoothed out characteristic point, comprising:
It is carried out using coordinate of the corresponding global change's matrix of each frame picture to corresponding characteristic point in the short track of characteristic point
Transformation, obtains each transformed coordinate of characteristic point in the short track of characteristic point；
The transformed coordinate of characteristic point each in the short track of characteristic point is combined, the short rail of characteristic point of precondition is obtained
Mark；
B-spline curves fitting is carried out to the short track of the characteristic point of the precondition, obtains the short track of smoothed out characteristic point.
Further, the B-spline curves fitting function is
Wherein PiFor control point, Ni,pIt (u) is p B-spline basic function.
Further, B-spline basic function is
Wherein k is B-spline power, and u is node, and i is the sequence frame number of B-spline.
In the present invention, by extracting the characteristic point in the every frame picture of video to be stablized, connection obtains the rail of each characteristic point
Mark and path length, and binding characteristic locus of points length threshold is classified, and the long track of characteristic point and the short rail of characteristic point are obtained
Every frame figure is calculated based on the long track of smoothed out characteristic point that the long track of characteristic point and low-pass filtered processing obtain in mark
The corresponding global change's matrix of piece, in conjunction with the corresponding global change's matrix of every frame picture, the short track of characteristic point and smoothing processing
Means obtain smoothed out short track, using more plane optimizing methods to each feature point trajectory and characteristic point smooth track
It is calculated, obtains the corresponding object transformation matrix of every frame picture, successively treat each pixel in the every frame picture of stable video
It is coordinately transformed using corresponding object transformation matrix, obtains stable UAV Video, track of the present invention to characteristic point
Classification, the calculating to the corresponding global change's matrix of every frame picture allows every frame picture using different transformation rules
Its movement is described, the movement of different zones can be accurately described, the present invention combines the use of more plane optimizing methods, Neng Gouyi
Determine degree and solves the influence of unmanned plane low-latitude flying scene roller shutter effect and CMOS camera high-frequency vibration scene to video, from
And the stablizing effect of UAV Video is improved, improve the availability of UAV Video.
The second aspect of the invention is to provide a kind of UAV Video stabilising arrangement of low-latitude flying scene, comprising:
Extraction module, for extracting the characteristic point in the every frame picture of video to be stablized, connection obtains the rail of each characteristic point
Mark and path length；
Comparison module obtains respectively for the path length of characteristic point to be compared with feature point trajectory length threshold
The long track of characteristic point and the short track of characteristic point；
Module is filtered, is smoothed, is obtained using low-pass filtering for the coordinate value to the long track of characteristic point
The smoothed out long track of characteristic point；
First computing module, for carrying out more plane optimizings to the long track of characteristic point and the long track of smoothed out characteristic point
It calculates, obtains the corresponding global change's matrix of every frame picture；
Conversion module, for being become using global change's matrix to the coordinate of corresponding characteristic point in the short track of characteristic point
It changes and smoothing processing, obtains the short track of smoothed out characteristic point；
Second computing module, for being carried out to the smoothed out long track of characteristic point and the short track of smoothed out characteristic point
Merging obtains characteristic point smooth track, is carried out using more plane optimizing methods to each feature point trajectory and characteristic point smooth track
It calculates, obtains the corresponding object transformation matrix of every frame picture；
Displacement compensation module is become for successively treating each pixel in the every frame picture of stable video using corresponding target
It changes matrix to be coordinately transformed, obtains stable UAV Video.
Further, first computing module includes:
It is corresponding each to obtain picture described in the long track of each characteristic point for being directed to every frame picture for first acquisition unit
The coordinate of a characteristic point；
Second acquisition unit, for obtaining the corresponding each characteristic point of picture described in the long track of smoothed out characteristic point
Coordinate；
Computing unit, using more plane optimizing methods to the corresponding each feature of picture described in the long track of each characteristic point
The coordinate of the corresponding each characteristic point of picture described in the coordinate of point and the long track of smoothed out characteristic point is calculated, and is obtained
To the corresponding global change's matrix of the picture.
Further, the conversion module includes:
Converter unit, for using the corresponding global change's matrix of each frame picture to corresponding feature in the short track of characteristic point
The coordinate of point is converted, and each transformed coordinate of characteristic point in the short track of characteristic point is obtained；
Assembled unit obtains pre- steady for being combined to the transformed coordinate of characteristic point each in the short track of characteristic point
The short track of fixed characteristic point；
Curve matching unit carries out B-spline curves fitting for the short track of characteristic point to the precondition, obtains smooth
The short track of characteristic point afterwards.
Further, the B-spline curves fitting function is
Wherein PiFor control point, Ni,pIt (u) is p B-spline basic function.
Further, B-spline basic function is
Wherein k is B-spline power, and u is node, i B
The sequence frame number of batten.
In the present invention, by extracting the characteristic point in the every frame picture of video to be stablized, connection obtains the rail of each characteristic point
Mark, and binding characteristic locus of points length threshold is classified, and the long track of characteristic point and the short track of characteristic point are obtained, and is based on characteristic point
The corresponding global change of every frame picture is calculated in the long track of smoothed out characteristic point that long track and low-pass filtered processing obtain
Matrix is changed, in conjunction with the corresponding global change's matrix of every frame picture, the short track of characteristic point and smoothing processing means, after obtaining smoothly
Short track, each feature point trajectory and characteristic point smooth track are calculated using more plane optimizing methods, obtained every
The corresponding object transformation matrix of frame picture, each pixel successively treated in the every frame picture of stable video are become using corresponding target
It changes matrix to be coordinately transformed, obtains stable UAV Video, classification of the present invention to the track of characteristic point, to every frame picture
The calculating of corresponding global change's matrix allows every frame picture to describe its movement, Neng Goujing using different transformation rules
The movement of different zones really is described, the present invention combines the use of more plane optimizing methods, can solve unmanned plane to a certain degree
The influence of low-latitude flying scene roller shutter effect and CMOS camera high-frequency vibration scene to video, to improve UAV Video
Stablizing effect, improve the availability of UAV Video.
Detailed description of the invention
Fig. 1 is the flow chart of UAV Video antihunt means one embodiment of low-latitude flying scene provided by the invention；
Fig. 2 is the process of another embodiment of the UAV Video antihunt means of low-latitude flying scene provided by the invention
Figure；
Fig. 3 is the process of another embodiment of the UAV Video antihunt means of low-latitude flying scene provided by the invention
Figure；
Fig. 4 is the structural representation of UAV Video stabilising arrangement one embodiment of low-latitude flying scene provided by the invention
Figure；
Fig. 5 is that the structure of another embodiment of the UAV Video stabilising arrangement of low-latitude flying scene provided by the invention is shown
It is intended to；
Fig. 6 is that the structure of another embodiment of the UAV Video stabilising arrangement of low-latitude flying scene provided by the invention is shown
It is intended to.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is
A part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
Every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
Fig. 1 is the flow chart of UAV Video antihunt means one embodiment of low-latitude flying scene provided by the invention,
As shown in Figure 1, comprising:
101, the characteristic point in the every frame picture of video to be stablized is extracted, connection obtains the track and track of each characteristic point
Length.
Specifically, angle point tracking can be used, is measurement with interframe gray scale difference quadratic sum, obtains wait stablize in video
Characteristic point and its coordinate information, and the same characteristic features point in each frame picture is associated, obtains the track of each characteristic point
And path length.
102, the path length of characteristic point is compared with feature point trajectory length threshold, obtains the long rail of characteristic point respectively
Mark and the short track of characteristic point.
Specifically, by after the trajectory map to X-Y coordinate of characteristic point, it is long path length can be greater than feature point trajectory
The feature point trajectory of degree threshold tau is determined as the long track T of characteristic pointL, path length is less than to the spy of feature point trajectory length threshold τ
The sign locus of points is determined as the short track T of characteristic pointS。
103, the coordinate value of the long track of characteristic point is smoothed using low-pass filtering, obtains smoothed out characteristic point
Long track.
It specifically, can be using one-dimensional Gaussian filter or track filter etc. to characteristic point in the long track of characteristic point
Abscissa track and ordinate track carry out one-dimensional low-pass filtering respectively, by the position for estimating characteristic point abscissa smooth track
And the position of ordinate smooth track, the long track of stable characteristic point is obtained, the stable long track of characteristic point is determined as putting down
The long track T of characteristic point after cunningL'。
104, more plane optimizing calculating are carried out to the long track of characteristic point and the long track of smoothed out characteristic point, obtains every frame
The corresponding global change's matrix of picture.
Wherein, the UAV Video stabilising arrangement of low-latitude flying scene can use more plane optimizing methods by every frame picture
Multiple planes are divided into, and it is every to combine characteristic point coordinate and corresponding smoothed out characteristic point coordinate in each plane to calculate
Transformation matrix set is determined as the transformation relation between picture and Target Photo to be stablized by the transformation matrix of one plane.
105, it is converted and is put down using coordinate of global change's matrix to corresponding characteristic point in the short track of characteristic point
Sliding processing, obtains the short track of smoothed out characteristic point.
Specifically, the UAV Video stabilising arrangement of low-latitude flying scene can use the corresponding global change of each frame picture
Matrix converts the coordinate for the characteristic point that each frame picture in the short track of characteristic point is included, and the characteristic point for obtaining precondition is short
The short track fitting of the characteristic point of precondition is the short track of smoothed out characteristic point by track, the method for then using curve matching
TS'。
106, the long track of smoothed out characteristic point and the short track of smoothed out characteristic point are merged to obtain characteristic point flat
Sliding rail mark calculates the track of each characteristic point and characteristic point smooth track using more plane optimizing methods, obtains every frame
The corresponding object transformation matrix of picture.
Specifically, the UAV Video stabilising arrangement of low-latitude flying scene obtains the long track T of characteristic pointL, smoothed out spy
The long track T of sign pointL', the short track T of characteristic pointS, the smoothed out short track T of characteristic pointS' later, it can be by the long track T of characteristic pointL
With the short track T of characteristic pointSCombination is characterized track T a littleC, by the long track T of smoothed out characteristic pointL' and smoothed out characteristic point
Short track TS' group is combined into smoothed out feature point trajectory TC', the corresponding target of every frame picture is obtained using more plane optimizing methods
Transformation matrix M'={ H1',H2'...Hk-1',Hk', wherein Hk' be each plane transformation matrix.
107, each pixel successively treated in the every frame picture of stable video is sat using corresponding object transformation matrix
Mark transformation, obtains stable UAV Video.
Wherein, if the coordinate of all pixels point is matrix in every frame picture
The coordinate of all pixels point is P'=M'P after then converting, and each pixel is mapped to new coordinate by coordinate P'
System obtains the picture after a frame is stablized.Above-mentioned transformation will be successively carried out wait stablize each frame picture in video, can be obtained after stablizing
Sequence of pictures；Sequence of pictures after stabilization is spliced by serial number, stable video can be obtained.
In the present embodiment, by extracting the characteristic point in the every frame picture of video to be stablized, connection obtains each characteristic point
Track and path length, and binding characteristic locus of points length threshold is classified, and obtains the long track of characteristic point and characteristic point is short
Track is calculated based on the long track of characteristic point and the low-pass filtered long track of characteristic point handled after obtained filtering
The corresponding global change's matrix of every frame picture, in conjunction with the corresponding global change's matrix of every frame picture, the short track of characteristic point and song
Line fitting means obtain smoothed out short track, using more plane optimizing methods to the track of each characteristic point and it is smooth after
Feature point trajectory calculated, obtain the corresponding object transformation matrix of every frame picture, become using the corresponding target of every frame picture
The coordinate progress conversion process that matrix treats each pixel in the corresponding picture of stable video is changed, obtains stablizing video, the present invention
Classification to the track of characteristic point, the calculating to the corresponding global change's matrix of every frame picture, uses every frame picture
Different transformation rules describes its movement, can accurately describe its movement, and the present invention combines the use of more plane optimizing methods,
Unmanned plane low-latitude flying scene roller shutter effect and CMOS camera high-frequency vibration scene can be solved to a certain degree to the shadow of video
It rings, to improve the stablizing effect of low-latitude flying scene UAV Video, improves the availability of UAV Video.
Fig. 2 is the flow chart of another embodiment of UAV Video antihunt means provided by the invention, as shown in Fig. 2, In
On the basis of embodiment illustrated in fig. 1, step 104 be can specifically include:
1041, it is directed to every frame picture, obtains the coordinate of the corresponding each characteristic point of picture in the long track of each characteristic point.
1042, the coordinate of the corresponding each characteristic point of picture described in the smoothed out long track of characteristic point is obtained.
1043, using more plane optimizing methods to the corresponding each characteristic point of picture described in the long track of each characteristic point
The coordinate of the corresponding each characteristic point of picture described in coordinate and the long track of smoothed out characteristic point is calculated, and institute is obtained
State the corresponding global change's matrix of picture.
Specifically, the UAV Video stabilising arrangement of low-latitude flying scene specifically can be first by all spies in every frame picture
Levy point Pk,i, with four angle point V of each gridkLinear interpolation indicate that the linear interpolation coefficient of four angle points is expressed as
ωk, wherein k is grid serial number, and i is characteristic point serial number in the grid, is passed throughIt can be by four angle points and characteristic point
PkAcquire linear interpolation coefficientAt this timeAnd PkFor it is known that optimization after four angle point Vk" it is unknown.Energy term can be with table
It is shown as:
Wherein k is grid serial number, and i is the grid characteristic point serial number, Vk" it is angular coordinate after stablizing in k-th grid.
Keep this energy term minimum, the smallest V of the sum of all interpolation feature points and smooth features point Euclidean distance can be obtainedk" result.
In addition, each grid can be divided into two triangles, each triangle again can be with similitude come table
Up to its deformation.If weBe set to a right angled triangle right angle electrical and two vertex, then haveWhereinIndicate the ratio between right-angle side side length, R indicates rotating vector.Now all triangles are combined and are counted
It calculates, energy term can be obtained:
WhereinIt is three angle points in k-th of grid after stabilization, s is all triangles after segmentation
Shape combines two energy term optimizations:
E=Ed+γEt,
Wherein γ is the weight of energy term.By minimizing energy type, then it can be concluded that four angle points of each grid are sat
Mark Vk'。
At this time according to the original angular coordinate V of each gridkWith stablize angular coordinate Vk', according to identity transformation matrix rule
P'=HP can calculate the transformation matrix set M={ H of corresponding each grid1,H2...Hk-1,Hk, which is global become
Change matrix.
Fig. 3 is the process of another embodiment of the UAV Video antihunt means of low-latitude flying scene provided by the invention
Figure, as shown in figure 3, on the basis of embodiment shown in Fig. 1, due to there was only TLAnd TL' the calculating of global change's matrix is participated in, this is complete
Office's transformation matrix is still undesirable for processing image periphery and the sparse position of characteristic point, can be carried out using the short track of characteristic point
Supplement.Therefore, step 105 can specifically include:
1051, using the corresponding global change's matrix of each frame picture to the coordinate of corresponding characteristic point in the short track of characteristic point
It is converted, obtains each transformed coordinate of characteristic point in the short track of characteristic point.
It specifically, can be by the short track T of characteristic point after obtaining the corresponding global change's matrix of each frame pictureSIn each frame picture
Including the coordinate of characteristic point substitute into the corresponding global change's matrix of each frame picture.Characteristic point coordinate is indicated with P, then spy can be obtained
Coordinate P "=M*P after each characteristic point is stablized in the short track of sign point.
1052, the transformed coordinate of characteristic point each in the short track of characteristic point is combined, obtains the feature of precondition
The short track of point.
Wherein, the coordinate P " after the same characteristic features point in each frame picture being stablized is connected as feature point trajectory with the time, then
The short track of the characteristic point of available precondition, is expressed as TS”。
1053, B-spline curves fitting is carried out to the short track of the characteristic point of precondition, obtains the short rail of smoothed out characteristic point
Mark.
Wherein, track T short to the characteristic point of each preconditionS" carry out B-spline curves fitting, B-spline curves fitting function
ForWherein PiFor control point, Ni,pIt (u) is p B-spline basic function.
Wherein, B-spline basic function is
Wherein k is B-spline power, and u is node, and i is the sequence frame number of B-spline.To smoothed out B-spline curves in integer
Frame is sampled, and is linked as smooth track in the point that each frame obtains, as the smooth short track T of characteristic pointS'。
In the present embodiment, by extracting the characteristic point in the every frame picture of video to be stablized, connection obtains each characteristic point
Track, and binding characteristic locus of points length threshold is classified, and the long track of characteristic point and the short track of characteristic point is obtained, for every frame
Picture obtains the coordinate of the corresponding each characteristic point of picture in the long track of each characteristic point, obtains the long rail of smoothed out characteristic point
The coordinate of the corresponding each characteristic point of picture described in mark, using more plane optimizing methods to described in the long track of each characteristic point
The corresponding each characteristic point of picture described in the coordinate of the corresponding each characteristic point of picture and the long track of smoothed out characteristic point
Coordinate calculated, the corresponding global change's matrix of the picture is obtained, using the corresponding global change's matrix of each frame picture
The coordinate of corresponding characteristic point in the short track of characteristic point is converted, is obtained in the short track of characteristic point after each feature point transformation
Coordinate, the transformed coordinate of characteristic point each in the short track of characteristic point is combined, the short rail of characteristic point of precondition is obtained
Mark carries out B-spline curves fitting to the short track of the characteristic point of precondition, obtains the short track of smoothed out characteristic point, and use is mostly flat
Face optimization method calculates each feature point trajectory and characteristic point smooth track, obtains the corresponding target of every frame picture and becomes
Matrix is changed, each pixel successively treated in the every frame picture of stable video carries out coordinate change using corresponding object transformation matrix
It changes, obtains stable UAV Video, classification of the present invention to the track of characteristic point, global change's square corresponding to every frame picture
The calculating of battle array allows every frame picture to describe its movement using different transformation rules, can accurately describe different zones
Movement, the present invention combine more plane optimizing methods use, unmanned plane low-latitude flying scene roller shutter can be solved to a certain degree
The influence of effect and CMOS camera high-frequency vibration scene to video improves nothing to improve the stablizing effect of UAV Video
The availability of man-machine video.
Those of ordinary skill in the art will appreciate that: realize that all or part of the steps of above-mentioned each method embodiment can lead to
The relevant hardware of program instruction is crossed to complete.Program above-mentioned can be stored in a computer readable storage medium.The journey
When being executed, execution includes the steps that above-mentioned each method embodiment to sequence；And storage medium above-mentioned include: ROM, RAM, magnetic disk or
The various media that can store program code such as person's CD.
Fig. 4 is the structural representation of UAV Video stabilising arrangement one embodiment of low-latitude flying scene provided by the invention
Figure, as shown in Figure 4, comprising:
Extraction module 41, for extracting the characteristic point in the every frame picture of video to be stablized, connection obtains each characteristic point
Track and path length；
Comparison module 42 obtains respectively for the path length of characteristic point to be compared with feature point trajectory length threshold
Take the long track of characteristic point and the short track of characteristic point；
Module 43 is filtered, is smoothed, is obtained using low-pass filtering for the coordinate value to the long track of characteristic point
To the long track of smoothed out characteristic point；
First computing module 44, it is excellent for carrying out more planes to the long track of characteristic point and the long track of smoothed out characteristic point
Change and calculate, obtains the corresponding global change's matrix of every frame picture；
Conversion module 45, for being carried out using coordinate of global change's matrix to corresponding characteristic point in the short track of characteristic point
Transformation and smoothing processing, obtain the short track of smoothed out characteristic point；
Second computing module 46, for being closed to the long track of smoothed out characteristic point and the short track of smoothed out characteristic point
And characteristic point smooth track is obtained, each feature point trajectory and characteristic point smooth track are counted using more plane optimizing methods
It calculates, obtains the corresponding object transformation matrix of every frame picture；
Displacement compensation module 47, for successively treating each pixel in the every frame picture of stable video using corresponding target
Transformation matrix is coordinately transformed, and obtains stable UAV Video.
The UAV Video stabilising arrangement of low-latitude flying scene provided by the invention can be the view being mounted on unmanned plane
Frequency stabilizer perhaps video stabilization software or can be the video stabilization software being mounted on background server.
Specifically, extraction module 41 can use angle point tracking, with interframe gray scale difference quadratic sum be measurement, obtain to
Stablize the characteristic point and its coordinate information in video, and the same characteristic features point in each frame picture is associated, obtains each spy
Levy the track of point.
Path length can be greater than characteristic point rail for after the trajectory map to X-Y coordinate of characteristic point by comparison module 42
The feature point trajectory of mark length threshold τ is determined as the long track T of characteristic pointL, path length is less than feature point trajectory length threshold τ
Feature point trajectory be determined as the short track T of characteristic pointS。
Module 43 is filtered specifically can be long to characteristic point using one-dimensional Gaussian filter or long track filter etc.
The abscissa track and ordinate track of characteristic point carry out one-dimensional low-pass filtering respectively in track, by estimating characteristic point abscissa
The position of smooth track and the position of ordinate smooth track, obtain the long track of stable characteristic point, by stable characteristic point
Long track is determined as the long track T of smoothed out characteristic pointL'。
The specific available long track T of characteristic point of second computing module 46L, the smoothed out long track T of characteristic pointL', feature
The short track T of pointS, the smoothed out short track T of characteristic pointS' later, it can be by the long track T of characteristic pointLWith the short track T of characteristic pointSGroup
It is combined into the track T of characteristic pointC, by the long track T of smoothed out characteristic pointL' and the short track T of smoothed out characteristic pointS' group be combined into it is flat
Feature point trajectory T after cunningC', the corresponding object transformation matrix M'={ H of every frame picture is obtained using more plane optimizing methods1',
H2'...Hk-1',Hk'}。
It needs to be illustrated, the coordinate of all pixels point in every frame picture can be set as matrix
The coordinate of all pixels point is P'=M'P after then converting, and each pixel is mapped to new coordinate by coordinate P'
System obtains the picture after a frame is stablized.Above-mentioned transformation will be successively carried out wait stablize each frame picture in video, can be obtained after stablizing
Sequence of pictures；Sequence of pictures after stabilization is spliced by serial number, stable video can be obtained.
In the present embodiment, by extracting the characteristic point in the every frame picture of video to be stablized, connection obtains each characteristic point
Track and path length, and binding characteristic locus of points length threshold is classified, and obtains the long track of characteristic point and characteristic point is short
Track is calculated based on the long track of characteristic point and the low-pass filtered long track of characteristic point handled after obtained filtering
The corresponding global change's matrix of every frame picture, in conjunction with the corresponding global change's matrix of every frame picture, the short track of characteristic point and song
Line fitting means obtain smoothed out short track, using more plane optimizing methods to the track of each characteristic point and it is smooth after
Feature point trajectory calculated, obtain the corresponding object transformation matrix of every frame picture, become using the corresponding target of every frame picture
The coordinate progress conversion process that matrix treats each pixel in the corresponding picture of stable video is changed, obtains stablizing video, the present invention
Classification to the track of characteristic point, the calculating to the corresponding global change's matrix of every frame picture, uses every frame picture
Different transformation rules describes its movement, can accurately describe its movement, and the present invention combines the use of more plane optimizing methods,
Unmanned plane low-latitude flying scene roller shutter effect and CMOS camera high-frequency vibration scene can be solved to a certain degree to the shadow of video
It rings, to improve the stablizing effect of the UAV Video of low-latitude flying scene, improves the availability of UAV Video.
Further, on the basis of the embodiment shown in fig. 4, in conjunction with reference Fig. 5, first computing module 44 can be with
Include:
It is corresponding to obtain picture described in the long track of each characteristic point for being directed to every frame picture for first acquisition unit 441
The coordinate of each characteristic point；
Second acquisition unit 442, for obtaining the corresponding each feature of picture described in the long track of smoothed out characteristic point
The coordinate of point；
Computing unit 443, for corresponding to picture described in the long track of each characteristic point using more plane optimizing methods
The coordinate of the corresponding each characteristic point of picture described in the coordinate of each characteristic point and the long track of smoothed out characteristic point carries out
It calculates, obtains the corresponding global change's matrix of the picture.
Specifically, the UAV Video stabilising arrangement of low-latitude flying scene specifically can be first by all spies in every frame picture
Levy point Pk,i, with four angle point V of each gridkLinear interpolation indicate that the linear interpolation coefficient of four angle points is expressed as
ωk, wherein k is grid serial number, and i is characteristic point serial number in the grid, is passed throughIt can be by four angle points and characteristic point
PkAcquire linear interpolation coefficientAt this timeAnd PkFor it is known that optimization after four angle point Vk" it is unknown.Energy term can be with
It indicates are as follows:
Wherein k is grid serial number, and i is the grid characteristic point serial number, Vk" it is angular coordinate after stablizing in k-th grid.
Keep this energy term minimum, the smallest V of the sum of all interpolation feature points and smooth features point Euclidean distance can be obtainedk" result.
In addition, each grid can be divided into two triangles, each triangle again can be with similitude come table
Up to its deformation.If weBe set to a right angled triangle right angle electrical and two vertex, then haveWhereinIndicate the ratio between right-angle side side length, R indicates rotating vector.Now all triangles are combined and are counted
It calculates, energy term can be obtained:
Wherein s is all triangles after segmentation, combines two energy term optimizations:
E=Ed+γEt,
Wherein γ is the weight of two energy terms.By minimizing energy type, then it can be concluded that four angle points of each grid
Coordinate Vk'。
At this time according to the original angular coordinate V of each gridkWith stablize angular coordinate Vk', according to identity transformation matrix rule
P'=HP can calculate the transformation matrix set M={ H of corresponding each grid1,H2...Hk-1,Hk, which is global become
Change matrix.
Further, on the basis of the embodiment shown in fig. 4, in conjunction with reference Fig. 6, the conversion module 45 can also be wrapped
It includes:
Converter unit 451, for using the corresponding global change's matrix of each frame picture to corresponding in the short track of characteristic point
The coordinate of characteristic point is converted, and each transformed coordinate of characteristic point in the short track of characteristic point is obtained；
Assembled unit 452 obtains pre- for being combined to the transformed coordinate of characteristic point each in the short track of characteristic point
The short track of stable characteristic point；
Curve matching unit 453 carries out B-spline curves fitting for the short track of characteristic point to the precondition, obtains
The smoothed out short track of characteristic point.
Wherein, after obtaining the corresponding global change's matrix of each frame picture, converter unit 451 can be by the short track T of characteristic pointSIn
The coordinate for the characteristic point that each frame picture includes substitutes into the corresponding global change's matrix of each frame picture.Characteristic point coordinate is indicated with P, then
Coordinate P "=M*P after each characteristic point is stablized in the short track of characteristic point can be obtained.Assembled unit 452 is by the phase in each frame picture
Coordinate P " after stablizing with characteristic point is connected as feature point trajectory with the time, then the short track of the characteristic point of available precondition,
It is expressed as TS”。
Wherein, track T short to the characteristic point of each preconditionS" carry out B-spline curves fitting, B-spline curves fitting function
MeetWherein PiFor control point, Ni,pIt (u) is p B-spline basic function.
Wherein, B-spline basic function is
Wherein k is B-spline power, and u is node, and i is the sequence frame number of B-spline.To smoothed out B-spline curves in integer
Frame is sampled, and is linked as smooth track in the point that each frame obtains, as the smooth short track T of characteristic pointS'。
In the present embodiment, by extracting the characteristic point in the every frame picture of video to be stablized, connection obtains each characteristic point
Track, and binding characteristic locus of points length threshold is classified, and the long track of characteristic point and the short track of characteristic point is obtained, for every frame
Picture obtains the coordinate of the corresponding each characteristic point of picture in the long track of each characteristic point, obtains the long rail of smoothed out characteristic point
The coordinate of the corresponding each characteristic point of picture described in mark, using more plane optimizing methods to described in the long track of each characteristic point
The corresponding each characteristic point of picture described in the coordinate of the corresponding each characteristic point of picture and the long track of smoothed out characteristic point
Coordinate calculated, the corresponding global change's matrix of the picture is obtained, using the corresponding global change's matrix of each frame picture
The coordinate of corresponding characteristic point in the short track of characteristic point is converted, is obtained in the short track of characteristic point after each feature point transformation
Coordinate, the transformed coordinate of characteristic point each in the short track of characteristic point is combined, the short rail of characteristic point of precondition is obtained
Mark carries out B-spline curves fitting to the short track of the characteristic point of precondition, obtains the short track of smoothed out characteristic point, and use is mostly flat
Face optimization method calculates each feature point trajectory and characteristic point smooth track, obtains the corresponding target of every frame picture and becomes
Matrix is changed, each pixel successively treated in the every frame picture of stable video carries out coordinate change using corresponding object transformation matrix
It changes, obtains stable UAV Video, classification of the present invention to the track of characteristic point, global change's square corresponding to every frame picture
The calculating of battle array allows every frame picture to describe its movement using different transformation rules, can accurately describe different zones
Movement, the present invention combine more plane optimizing methods use, unmanned plane low-latitude flying scene roller shutter can be solved to a certain degree
The influence of effect and CMOS camera high-frequency vibration scene to video improves nothing to improve the stablizing effect of UAV Video
The availability of man-machine video.
Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Pipe present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: its according to
So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into
Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme.
Claims (6)
1. a kind of UAV Video antihunt means of low-latitude flying scene characterized by comprising
The characteristic point in the every frame picture of video to be stablized is extracted, connection obtains track and the path length of each characteristic point；
The path length of characteristic point is compared with feature point trajectory length threshold, obtains the long track of characteristic point and feature respectively
The short track of point；
The coordinate value of the long track of characteristic point is smoothed using low-pass filtering, obtains the long track of smoothed out characteristic point；
More plane optimizing calculating are carried out to the long track of characteristic point and the long track of smoothed out characteristic point, it is corresponding to obtain every frame picture
Global change's matrix；
Transformation and smoothing processing are carried out using coordinate of global change's matrix to corresponding characteristic point in the short track of characteristic point, obtained
To the short track of smoothed out characteristic point；
The smoothed out long track of characteristic point and the short track of smoothed out characteristic point are merged to obtain the smooth rail of characteristic point
Mark calculates each feature point trajectory and characteristic point smooth track using more plane optimizing methods, obtains every frame picture pair
The object transformation matrix answered；
The each pixel successively treated in the every frame picture of stable video is coordinately transformed using corresponding object transformation matrix, is obtained
To stable UAV Video.
2. the UAV Video antihunt means of low-latitude flying scene according to claim 1, which is characterized in that described to spy
The sign long track of point and the long track of smoothed out characteristic point carry out more plane optimizing calculating, obtain the corresponding global change of every frame picture
Change matrix, comprising:
For every frame picture, the coordinate of the corresponding each characteristic point of picture described in the long track of each characteristic point is obtained；
Obtain the coordinate of the corresponding each characteristic point of picture described in the long track of smoothed out characteristic point；
Using more plane optimizing methods to the coordinate of the corresponding each characteristic point of picture described in the long track of each characteristic point, and
The coordinate of the corresponding each characteristic point of picture described in the smoothed out long track of characteristic point is calculated, and it is corresponding to obtain the picture
Global change's matrix.
3. the UAV Video antihunt means of low-latitude flying scene according to claim 1, which is characterized in that the use
Global change's matrix carries out transformation and smoothing processing to the coordinate of corresponding characteristic point in the short track of characteristic point, after obtaining smoothly
The short track of characteristic point, comprising:
It is converted using coordinate of the corresponding global change's matrix of each frame picture to corresponding characteristic point in the short track of characteristic point,
Obtain each transformed coordinate of characteristic point in the short track of characteristic point；
The transformed coordinate of characteristic point each in the short track of characteristic point is combined, the short track of characteristic point of precondition is obtained；
B-spline curves fitting is carried out to the short track of the characteristic point of the precondition, obtains the short track of smoothed out characteristic point；
Wherein, the B-spline curves fitting function is
Wherein PiFor control point, Ni,pIt (u) is p B-spline basic function；
Wherein, the B-spline basic function is
Wherein k is B-spline power, and u is node, and i is the sequence frame number of B-spline.
4. a kind of UAV Video stabilising arrangement of low-latitude flying scene characterized by comprising
Extraction module, for extracting the characteristic point in the every frame picture of video to be stablized, connection obtain the track of each characteristic point with
And path length；
Comparison module obtains feature for the path length of characteristic point to be compared with feature point trajectory length threshold respectively
The long track of point and the short track of characteristic point；
Module is filtered, is smoothed for the coordinate value to the long track of characteristic point using low-pass filtering, obtains smooth
The long track of characteristic point afterwards；
First computing module, by being carried out based on more plane optimizings to the long track of characteristic point and the long track of smoothed out characteristic point
It calculates, obtains the corresponding global change's matrix of every frame picture；
Conversion module, for use global change's matrix to the coordinate of corresponding characteristic point in the short track of characteristic point converted with
And smoothing processing, obtain the short track of smoothed out characteristic point；
Second computing module, for being merged to the smoothed out long track of characteristic point with the short track of smoothed out characteristic point
Characteristic point smooth track is obtained, each feature point trajectory and characteristic point smooth track are counted using more plane optimizing methods
It calculates, obtains the corresponding object transformation matrix of every frame picture；
Displacement compensation module, for successively treating each pixel in the every frame picture of stable video using corresponding object transformation square
Battle array is coordinately transformed, and obtains stable UAV Video.
5. the UAV Video stabilising arrangement of low-latitude flying scene according to claim 4, which is characterized in that described first
Computing module includes:
First acquisition unit obtains the corresponding each spy of picture described in the long track of each characteristic point for being directed to every frame picture
Levy the coordinate of point；
Second acquisition unit, for obtaining the seat of the corresponding each characteristic point of picture described in the long track of smoothed out characteristic point
Mark；
Computing unit, using more plane optimizing methods to the corresponding each characteristic point of picture described in the long track of each characteristic point
The coordinate of the corresponding each characteristic point of picture described in coordinate and the long track of smoothed out characteristic point is calculated, and institute is obtained
State the corresponding global change's matrix of picture.
6. the UAV Video stabilising arrangement of low-latitude flying scene according to claim 4, which is characterized in that the transformation
Module includes:
Converter unit, for using the corresponding global change's matrix of each frame picture to corresponding characteristic point in the short track of characteristic point
Coordinate is converted, and each transformed coordinate of characteristic point in the short track of characteristic point is obtained；
Assembled unit obtains precondition for being combined to the transformed coordinate of characteristic point each in the short track of characteristic point
The short track of characteristic point；
Curve matching unit carries out B-spline curves fitting for the short track of characteristic point to the precondition, obtains smoothed out
The short track of characteristic point.
Wherein, the B-spline curves fitting function is
Wherein PiFor control point, Ni,pIt (u) is p B-spline basic function.
Wherein, the B-spline basic function is
Wherein k is B-spline power, and u is node, and i is the sequence frame number of B-spline.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710198431.7A CN107135331B (en) | 2017-03-29 | 2017-03-29 | The UAV Video antihunt means and device of low-latitude flying scene |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710198431.7A CN107135331B (en) | 2017-03-29 | 2017-03-29 | The UAV Video antihunt means and device of low-latitude flying scene |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107135331A CN107135331A (en) | 2017-09-05 |
| CN107135331B true CN107135331B (en) | 2019-12-03 |
Family
ID=59715994
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710198431.7A Active CN107135331B (en) | 2017-03-29 | 2017-03-29 | The UAV Video antihunt means and device of low-latitude flying scene |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107135331B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112804444B (en) * | 2020-12-30 | 2022-08-23 | 影石创新科技股份有限公司 | Video processing method and device, computing equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102780846A (en) * | 2012-07-11 | 2012-11-14 | 清华大学 | Electronic image stabilization method based on inertial navigation information |
| CN103093481A (en) * | 2013-01-28 | 2013-05-08 | 中国科学院上海微系统与信息技术研究所 | Moving object detection method under static background based on watershed segmentation |
| CN103414853A (en) * | 2013-07-26 | 2013-11-27 | 东华大学 | Device and method for stabilizing video image sequence capable of doing multi-degree of freedom movement in real time |
| KR101638628B1 (en) * | 2015-03-06 | 2016-07-12 | 서울과학기술대학교 산학협력단 | a Image stabilization device of a camera |
| CN105761233A (en) * | 2014-12-15 | 2016-07-13 | 南京理工大学 | FPGA-based real-time panoramic image mosaic method |
| JP2017021430A (en) * | 2015-07-07 | 2017-01-26 | Kddi株式会社 | Panoramic video data processing device, processing method, and program |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI456984B (en) * | 2011-10-24 | 2014-10-11 | Univ Nat Central | Video stabilization system and method thereof |
| US9277129B2 (en) * | 2013-06-07 | 2016-03-01 | Apple Inc. | Robust image feature based video stabilization and smoothing |
-
2017
- 2017-03-29 CN CN201710198431.7A patent/CN107135331B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102780846A (en) * | 2012-07-11 | 2012-11-14 | 清华大学 | Electronic image stabilization method based on inertial navigation information |
| CN103093481A (en) * | 2013-01-28 | 2013-05-08 | 中国科学院上海微系统与信息技术研究所 | Moving object detection method under static background based on watershed segmentation |
| CN103414853A (en) * | 2013-07-26 | 2013-11-27 | 东华大学 | Device and method for stabilizing video image sequence capable of doing multi-degree of freedom movement in real time |
| CN105761233A (en) * | 2014-12-15 | 2016-07-13 | 南京理工大学 | FPGA-based real-time panoramic image mosaic method |
| KR101638628B1 (en) * | 2015-03-06 | 2016-07-12 | 서울과학기술대학교 산학협력단 | a Image stabilization device of a camera |
| JP2017021430A (en) * | 2015-07-07 | 2017-01-26 | Kddi株式会社 | Panoramic video data processing device, processing method, and program |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107135331A (en) | 2017-09-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Joint filtering of intensity images and neuromorphic events for high-resolution noise-robust imaging | |
| Shu et al. | Feature-metric loss for self-supervised learning of depth and egomotion | |
| CN107862704B (en) | Target tracking method and system and holder camera used by same | |
| CN111275626B (en) | Video deblurring method, device and equipment based on ambiguity | |
| JP5744217B2 (en) | Method and system for processing video for stabilization and retargeting | |
| CN101924874B (en) | Matching block-grading realtime electronic image stabilizing method | |
| CN103139568B (en) | Based on the Video Stabilization method of degree of rarefication and fidelity constraint | |
| CN109389555B (en) | Panoramic image splicing method and device | |
| US11748894B2 (en) | Video stabilization method and apparatus and non-transitory computer-readable medium | |
| CN107749987B (en) | Digital video image stabilization method based on block motion estimation | |
| CN106878612B (en) | A kind of video stabilizing method based on the optimization of online total variation | |
| KR20150132846A (en) | Cascaded camera motion estimation, rolling shutter detection, and camera shake detection for video stabilization | |
| CN110796010A (en) | Video image stabilization method combining optical flow method and Kalman filtering | |
| CN110175011B (en) | Panoramic image seamless splicing method | |
| CN102256061A (en) | Two-dimensional and three-dimensional hybrid video stabilizing method | |
| CN111798373A (en) | Rapid unmanned aerial vehicle image stitching method based on local plane hypothesis and six-degree-of-freedom pose optimization | |
| CN110992395A (en) | Image training sample generation method and device and motion tracking method and device | |
| Wang et al. | Video stabilization: A comprehensive survey | |
| CN114429191B (en) | Electronic anti-shake method, system and storage medium based on deep learning | |
| CN107360377B (en) | Vehicle-mounted video image stabilization method | |
| JP4935930B2 (en) | Image correction apparatus and image correction method | |
| CN110580715B (en) | Image alignment method based on illumination constraint and grid deformation | |
| Chen et al. | Pixstabnet: Fast multi-scale deep online video stabilization with pixel-based warping | |
| Huang et al. | Stablenet: semi-online, multi-scale deep video stabilization | |
| CN107135331B (en) | The UAV Video antihunt means and device of low-latitude flying scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |