CN103813056B - A kind of digital image stabilization method and device - Google Patents

A kind of digital image stabilization method and device Download PDF

Info

Publication number
CN103813056B
CN103813056B CN201210463920.8A CN201210463920A CN103813056B CN 103813056 B CN103813056 B CN 103813056B CN 201210463920 A CN201210463920 A CN 201210463920A CN 103813056 B CN103813056 B CN 103813056B
Authority
CN
China
Prior art keywords
point
vector
image
frame image
current frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201210463920.8A
Other languages
Chinese (zh)
Other versions
CN103813056A (en
Inventor
孙莉
潘华东
潘石柱
吴良健
田建国
周洪涛
张兴明
傅利泉
朱江明
吴军
吴坚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Dahua Technology Co Ltd
Original Assignee
Zhejiang Dahua Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Dahua Technology Co Ltd filed Critical Zhejiang Dahua Technology Co Ltd
Priority to CN201210463920.8A priority Critical patent/CN103813056B/en
Publication of CN103813056A publication Critical patent/CN103813056A/en
Application granted granted Critical
Publication of CN103813056B publication Critical patent/CN103813056B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Image Analysis (AREA)
  • Studio Devices (AREA)
  • Television Systems (AREA)

Abstract

The invention discloses a kind of digital image stabilization method and device, comprising: from the characteristic point of current frame image, determine that the specific characteristic point with the characteristic point characteristic matching of current reference two field picture, this characteristic point are meet the point presetting characteristic condition in image; Determine that the estimation of motion vectors point of this current frame image is in all specific characteristics point, other specific characteristic point except the specific characteristic point in partial moving picture block; This partial moving picture block is the image block comprising Moving Objects image in this current frame image; Based on the coordinate of this estimation of motion vectors point in this current frame image, and in this current reference frame image with the coordinate of the characteristic point of this estimation of motion vectors points correspondence, determine the compensating motion vector of this current frame image; Based on this compensating motion vector, steady picture is carried out to this current frame image and compensates.Adopt method provided by the invention and device, solve the problem of image steady picture weak effect.

Description

A kind of digital image stabilization method and device
Technical field
The present invention relates to video field, particularly relate to a kind of digital image stabilization method and device.
Background technology
The development experience of Video Stabilization technology mechanical steady picture, photorefractive crystals and electronic steady image three megastage.Machinery similarly is surely utilize stabilized platform to realize the stable of image sequence to compensate the relative motion of camera system on pedestal.Photorefractive crystals utilizes the compensating motion of some elements in optical system to realize the stable of image sequence.Compare with photorefractive crystals system with traditional machinery, electronic steady image system have volume little, lightweight, be easy to operation, flexibility is strong, energy consumption is low and high-intelligentization such as to process in real time at the advantage, so electronic image stabilizing is the important directions of steady picture technical development.
In prior art, conventional electronic image stabilization method is that i.e. global motion vector, using global motion vector as compensating motion vector based on the motion vector of Gray Projection method determination current frame image relative to current reference two field picture.In a lot of occasion, current frame image relative to current reference two field picture substantially without convergent-divergent, therefore ignore compensation contraction coefficient s, compensating motion vector only comprises compensation anglec of rotation horizontal component a, compensates anglec of rotation vertical component b, these four kinds of vectors of level compensating motion vector dx and VCP motion vector dy.
Adopt following formula to carry out steady picture to current frame image based on this compensating motion vector to compensate:
α=tan -1(b/a)；
x c out y c out = cos a - sin a sin a cos a × x c y c + dx dy ·
Wherein, α is for compensating the anglec of rotation; for the coordinate of a certain pixel in current frame image, for the coordinate of this pixel after steady picture compensates in image.
Visible, compensating motion vector plays conclusive effect to the steady of video image as effect.But, because actual environment complexity is various, Moving Objects image may be there is in video image, therefore, using not accurate enough as compensating motion vector relative to the global motion vector of current reference two field picture for the current frame image determined based on Gray Projection method, and then reducing the Stability and veracity of successive image compensation, steady picture effect is poor.
Summary of the invention
The embodiment of the present invention provides a kind of digital image stabilization method and device, in order to solve the steady problem poor as effect of the image existed in prior art.
The embodiment of the present invention provides a kind of digital image stabilization method, comprising:
From the characteristic point of current frame image, determine that the specific characteristic point with the characteristic point characteristic matching of current reference two field picture, described characteristic point are meet the point presetting characteristic condition in image;
Determine that the estimation of motion vectors point of described current frame image is in all specific characteristics point, other specific characteristic point except the specific characteristic point in partial moving picture block; Described partial moving picture block is the image block comprising Moving Objects image in described current frame image;
Based on the coordinate of the described estimation of motion vectors point in described current frame image, and in described current reference two field picture with the coordinate of the characteristic point of described estimation of motion vectors points correspondence, determine the compensating motion vector of described current frame image;
Based on described compensating motion vector, steady picture is carried out to described current frame image and compensates.
The embodiment of the present invention provides a kind of steady picture device, comprising:
Matching unit, for determining the specific characteristic point with the characteristic point characteristic matching of current reference two field picture from the characteristic point of current frame image, described characteristic point is meet the point presetting characteristic condition in image;
First determining unit, for determining that the estimation of motion vectors point of described current frame image is in all specific characteristics point, other specific characteristic point except the specific characteristic point in partial moving picture block; Described partial moving picture block is the image block comprising Moving Objects image in described current frame image;
Second determining unit, for the coordinate based on the described estimation of motion vectors point in described current frame image, with in described current reference two field picture with the coordinate of the characteristic point of described estimation of motion vectors points correspondence, determine the compensating motion vector of described current frame image;
Compensating unit, for based on described compensating motion vector, carries out steady picture to described current frame image and compensates.
Beneficial effect of the present invention comprises:
The method that the embodiment of the present invention provides, when carrying out the screening of estimation of motion vectors point, eliminate in current frame image the specific characteristic point comprised in the partial moving picture block of Moving Objects image, using other specific characteristic point as estimation of motion vectors point, can make the compensating motion vector of image and estimating more accurately, and then improve the Stability and veracity of successive image compensation, steady picture effect is better.
Accompanying drawing explanation
Accompanying drawing is used to provide a further understanding of the present invention, and forms a part for specification, is used from explanation the present invention, is not construed as limiting the invention with the embodiment of the present invention one.In the accompanying drawings:
The flow chart of the digital image stabilization method that Fig. 1 provides for the embodiment of the present invention;
The detail flowchart of the digital image stabilization method that Fig. 2 provides for the embodiment of the present invention;
The structural representation of the steady picture device that Fig. 3 provides for the embodiment of the present invention.
Embodiment
In order to provide the implementation improving image steady picture effect, embodiments provide a kind of digital image stabilization method and device, below in conjunction with Figure of description, the preferred embodiments of the present invention are described, be to be understood that, preferred embodiment described herein, only for instruction and explanation of the present invention, is not intended to limit the present invention.And when not conflicting, the embodiment in the application and the feature in embodiment can combine mutually.
The embodiment of the present invention provides a kind of digital image stabilization method, comprising:
Step 101, from the characteristic point of current frame image, determine that the specific characteristic point with the characteristic point characteristic matching of current reference two field picture, this characteristic point are meet in image to preset the point of characteristic condition.
Step 102, determine that the estimation of motion vectors point of this current frame image is in all specific characteristics point, other specific characteristic point except the specific characteristic point in partial moving picture block; This partial moving picture block is the image block comprising Moving Objects image in this current frame image.
Step 103, coordinate based on this estimation of motion vectors point in this current frame image, and in this current reference frame image with the coordinate of the characteristic point of this estimation of motion vectors points correspondence, determine the compensating motion vector of this current frame image.
Step 104, based on this compensating motion vector, steady picture is carried out to this current frame image and compensates.
Wherein, in step 103, based on the coordinate of the estimation of motion vectors point in current frame image, with in current reference two field picture with the coordinate of the characteristic point of estimation of motion vectors points correspondence, a kind of random sampling unification algorism that the embodiment of the present invention specifically can be adopted to propose is to determine the method for the compensating motion vector of current frame image.
Preferably, before execution step 104, the reliability of the compensating motion vector determined in step 103 is judged specifically whether can meet first by the compensating motion vector of the previous frame image of the compensating motion vector and current frame image that judge current frame image and preset condition of similarity and realize judging the reliability of compensating motion vector.The absolute difference of the corresponding vector that the various vectors that this first default condition of similarity can comprise for the compensating motion vector of current frame image comprise with the compensating motion vector of the previous frame image of current frame image respectively is all less than corresponding threshold value, also can be other condition of similarity.
When the compensating motion vector of the previous frame image of the compensating motion vector and current frame image of determining current frame image meets this first default condition of similarity, determine that the reliability of the compensating motion vector of current frame image is high, perform step 104, steady picture is carried out to current frame image and compensates; When the compensating motion vector of the previous frame image of the compensating motion vector and current frame image of determining current frame image does not meet this first default condition of similarity, determine that the reliability of the compensating motion vector of current frame image is low, do not perform step 104, cancel and steady picture compensation is carried out to current frame image.
But preset the reliability of condition of similarity to compensating motion vector by means of only first and judge, judged result may be not accurate enough.Therefore when the compensating motion vector of the previous frame image of the compensating motion vector and current frame image of determining current frame image do not meet this first preset condition of similarity time, directly cancelling and carrying out to current frame image scheme that steady picture compensates is not preferably scheme.
Preferably, can when the compensating motion vector of the previous frame image when the compensating motion vector and current frame image of determining current frame image meet this first default condition of similarity, carry out follow-up judgement by other condition again, determine whether to cancel and steady picture compensation is carried out to current frame image.
And, when the compensating motion vector of the previous frame image of the compensating motion vector and current frame image of determining current frame image does not meet the first default condition of similarity, upgrade the next frame image that current reference two field picture is current frame image or current frame image, as the reference frame image of next frame image.Also can work as current reference two field picture as when being greater than frequency threshold value with reference to the number of times of two field picture, upgrade the next frame image that current reference two field picture is current frame image or current frame image.
Above-mentioned digital image stabilization method, have ignored compensation contraction coefficient s, namely current frame image is applicable to relative to current reference two field picture substantially without the occasion of convergent-divergent, steady picture can be carried out to the output image of monitor video etc., the picture format of acquiescence is YUV color space form, if when carrying out video acquisition, the video image of acquisition is not YUV color space form, needs to carry out format conversion.And when carrying out video image acquisition, expansion collection should be carried out according to the steady size as compensation range to image surrounding.Below in conjunction with accompanying drawing, with specific embodiment, method provided by the invention and device are described in detail.
Figure 2 shows that the detail flowchart of the digital image stabilization method that the embodiment of the present invention provides, specifically comprise following treatment step:
Step 201, from the characteristic point of current frame image, determine the specific characteristic point with the characteristic point characteristic matching of current reference two field picture.
Wherein, characteristic point is specifically as follows the characteristic points such as angle point, sift characteristic point or surf characteristic point.Here be specifically described for angle point.
Corner Detection is carried out to current frame image and current reference two field picture, and respectively pyramid space delamination is carried out to current frame image and current reference two field picture, after calculating the characteristic vector of each angle point, the coupling of angle point is carried out according to characteristic vector, the appointment angle point with the corners Matching of current reference two field picture is determined from the angle point of current frame image, and the coordinate that the angle point recording successful match is right.Concrete Corner Detection matching process is all carry out under the gray level image of current frame image and current reference two field picture, preferably, before carrying out Corner Detection, can carry out noise reduction process to the gray level image of current frame image, this part is prior art, therefore is not described in detail in this.
Do not carry out more under news in reference frame image, the detection of current reference frame image angle point and characteristic vector calculate only once preserves.
Step 202, current frame image is divided at least a 3rd predetermined number image block, determine the partial moving picture block in a 3rd predetermined number image block, partial moving picture block is the image block comprising Moving Objects image in current frame image.
Preferably, current frame image can be divided into the image block of at least a 3rd predetermined number homalographic, choose and comprise a fairly large number of 3rd predetermined number image block of specific characteristic point, based on Gray Projection method determination global level motion vector, the local horizontal motion vector of each image block in overall situation vertical motion vector and the image block that is selected, partial vertical motion vector, determine that partial moving picture block is that the absolute difference of local horizontal motion vector and global level motion vector in a 3rd predetermined number image block is greater than differential horizontal threshold value, and/or the absolute difference of partial vertical motion vector and overall vertical motion vector is greater than the image block of differential vertical threshold value.Wherein, global level motion vector is the horizontal motion vector of current frame image relative to current reference two field picture; Overall situation vertical motion vector is the vertical motion vector of current frame image relative to current reference two field picture; Local horizontal motion vector is the horizontal motion vector of image block relative to the image block of same position in current reference two field picture of current frame image; Partial vertical motion vector is the vertical motion vector of image block relative to the image block of same position in current reference two field picture of current frame image.
Gray Projection method is the motion estimation algorithm utilizing ranks intensity profile to carry out computing cross-correlation.This algorithm utilizes the ranks Gray scale projection curve of image to do the motion vector of computing cross-correlation acquisition image.Below to calculate the vertical motion vector of current frame image relative to current reference two field picture, be illustrated concrete computational process.
First, project to current frame image and current reference two field picture respectively, projection formula is:
Cgray(j)=∑Cur(i,j)；
Cm=[∑Cgray(i,j)]/NC；
Cout(j)=Cgray(j)-Cm。
Wherein, the gray value sum of pixel that arranges for jth of Cgray (j);
Cur (i, j) is the gray value of the pixel of the i-th row jth row;
Cm is the mean value of the gray value sum of the pixel of all row;
The gray value sum that ∑ Cgray (i, j) is the pixel of all row and value;
NC is total columns;
The projection value that Cout (j) arranges for jth.
Then, then to by the projection value Cout obtained that projects to current frame image c(j), and by the projection value Cout obtained that projects to current reference two field picture rj () carries out cross-correlation calculation, the computing formula of cross-correlation function C (v) is:
C ( v ) = Σ j = 1 NC [ Cout r ( j + v ) - Cout c ( j + m ) ] 2 ·
Wherein, v is vertical motion vector;
M is the window of cross-correlation calculation search, and 1≤v≤2m+1.
Vertical motion vector v corresponding when making cross-correlation function C (v) value minimum is the vertical motion vector of current frame image relative to current reference two field picture, i.e. overall vertical motion vector.
Adopt said method can determine local horizontal motion vector, the partial vertical motion vector of each image block in global level motion vector and the image block that is selected successively.
Step 203, determine that the estimation of motion vectors point of current frame image is in all specific characteristics point, other specific characteristic point except the specific characteristic point in partial moving picture block.
Step 204, coordinate based on the estimation of motion vectors point in current frame image, with in current reference two field picture with the coordinate of the characteristic point of estimation of motion vectors points correspondence, determined the compensating motion vector of current frame image by random sampling unification algorism, be specially:
A, current frame image is divided into the image-region of at least the first predetermined number homalographic.
B, for each image-region in the image-region of the first predetermined number estimation of motion vectors point containing at least the second predetermined number, following steps are adopted to determine the consistent point motion vector in the region of this image-region:
Determine the point motion vector of each estimation of motion vectors point in this image-region; For each designated movement vector estimation point in this image-region, determine the quantity of the consistent point of this designated movement vector estimation point in other each estimation of motion vectors point, wherein, designated movement vector estimation point is the estimation of motion vectors point of the second predetermined number chosen from all estimation of motion vectors points this image-region, and the consistent point of this designated movement vector estimation point is the estimation of motion vectors point that the point motion vector of point motion vector and this designated movement vector estimation point meets that second presets condition of similarity; Determine the point motion vector of the designated movement vector estimation point that the maximum of the quantity of each self-corresponding consistent point of each designated movement vector estimation point is corresponding, the consistent point motion vector in the region as this image-region.
Preferably, if the image-region of the estimation of motion vectors point containing at least the second predetermined number is more than the first predetermined number, the image-region that the first more predetermined number of estimation of motion vectors point is individual is therefrom chosen.Due to the coordinate of estimation of motion vectors point in current frame image x c y c , With in current reference two field picture with the coordinate of the characteristic point of this estimation of motion vectors points correspondence x r y r , Meet following relationship:
x r y r = cos a ′ - sin a ′ sin a ′ cos a ′ × x c y c + dx ′ dy ′ ·
Wherein, α ' is the anglec of rotation; Dx' is horizontal motion vector; Dy ' is vertical motion vector.By α '=tan -1(b'/a') substitution obtains:
x c - y c 1 0 y c x c 0 1 a ′ b ′ dx ′ dy ′ = x r y r ·
Wherein, a' is anglec of rotation horizontal component; B ' is anglec of rotation vertical component.
The point motion vector of each estimation of motion vectors point can be determined based on above-mentioned formula a ′ b ′ dx ′ dy ′ ·
The absolute difference of the corresponding vector that the various vectors that the second default condition of similarity can comprise for the point motion vector of estimation of motion vectors point comprise with the point motion vector of designated movement vector estimation point respectively is all less than corresponding threshold value, also can be other condition of similarity.
C, for often kind of vector in the consistent point motion vector in the first predetermined number region, adopt following steps determine the compensation vector that this kind of vector is corresponding:
For each appointment vector in this kind of vector, determine the quantity of the consistent vector of this appointment vector in other each vector, wherein, appointment vector is the vector of the 3rd predetermined number chosen from the first predetermined number this kind of vector, and the consistent vector of this appointment vector is meet the vector that the 3rd presets condition of similarity with this appointment vector; Determine the mean value of the consistent vector of the appointment vector that the maximum of the quantity of each self-corresponding consistent vector of each appointment vector is corresponding, as the compensation vector that this kind of vector is corresponding.
D, based on compensation vector corresponding to various vector, obtain the compensating motion vector of current frame image.Namely the compensating motion vector of current frame image is made up of the compensation vector that four kinds of vectors are corresponding a b dx dy ·
Step 205, judge whether the absolute difference of the corresponding vector that the various vectors that the compensating motion vector of current frame image comprises comprise with the compensating motion vector of the previous frame image of current frame image respectively is all less than corresponding threshold value.
When the absolute difference determining the corresponding vector that the various vectors that the compensating motion vector of current frame image comprises comprise with the compensating motion vector of the previous frame image of current frame image is not respectively, when being all less than corresponding threshold value, enter step 206; When determining that the absolute difference of the corresponding vector that the various vectors that the compensating motion vector of current frame image comprises comprise with the compensating motion vector of the previous frame image of current frame image respectively is all less than corresponding threshold value, enter step 207.
Step 206, judge whether the maximum that each self-corresponding region of image-region of the first predetermined number estimation of motion vectors point containing at least the second predetermined number is unanimously put in quantity is less than amount threshold.
When determining each self-corresponding region of image-region of the first predetermined number estimation of motion vectors point containing at least the second predetermined number unanimously the maximum put in quantity being not less than amount threshold, enter step 207; When determining each self-corresponding region of image-region of the first predetermined number estimation of motion vectors point containing at least the second predetermined number unanimously the maximum put in quantity being less than amount threshold, terminate this flow process, steady picture is not carried out to current frame image and compensate.
If when determining the compensating motion vector of current frame image, be not determined by the random sampling unification algorism in step 204, then directly can not obtain each self-corresponding region of image-region in this step and unanimously put quantity, now, need to calculate in this step, computational methods, with in step 204, are not described in detail in this.
Step 207, based on this compensating motion vector, steady picture is carried out to current frame image and compensates.
Further, maximum compensation range should not be exceeded to the steady picture compensation of current frame image.
Obtain in scene at some video images, some factors such as illumination can constantly change, and therefore need to carry out real-time update to reference frame image.Be specifically as follows:
When the absolute difference determining the corresponding vector that the various vectors that the compensating motion vector of current frame image comprises comprise with the compensating motion vector of the previous frame image of current frame image is not respectively when being all less than corresponding threshold value, or, when current reference two field picture is greater than frequency threshold value as the number of times with reference to two field picture, upgrade the next frame image that current reference two field picture is current frame image or current frame image, as the reference frame image of next frame image.Further, when to upgrade current reference two field picture be the next frame image of current frame image, steady picture operation can not be carried out to the next frame image of current frame image, directly steady picture operation be carried out to next frame image again.
Adopt the method that the embodiment of the present invention provides, when carrying out the screening of estimation of motion vectors point, eliminate in current frame image the specific characteristic point comprised in the partial moving picture block of Moving Objects image, using other specific characteristic point as estimation of motion vectors point, can make the compensating motion vector of image and estimating more accurately, and its reliability is judged further to the compensating motion vector estimated, determine whether to compensate current frame image according to reliability, and then improve the Stability and veracity of successive image compensation, steady picture effect is better.
Based on same inventive concept, according to the digital image stabilization method that the above embodiment of the present invention provides, correspondingly, another embodiment of the present invention additionally provides steady picture device, and apparatus structure schematic diagram as shown in Figure 3, specifically comprises:
Matching unit 301, for determining the specific characteristic point with the characteristic point characteristic matching of current reference two field picture from the characteristic point of current frame image, this characteristic point is meet the point presetting characteristic condition in image;
First determining unit 302, for determining that the estimation of motion vectors point of this current frame image is in all specific characteristics point, other specific characteristic point except the specific characteristic point in partial moving picture block; This partial moving picture block is the image block comprising Moving Objects image in this current frame image;
Second determining unit 303, for the coordinate based on this estimation of motion vectors point in this current frame image, and in this current reference frame image with the coordinate of the characteristic point of this estimation of motion vectors points correspondence, determine the compensating motion vector of this current frame image;
Compensating unit 304, for based on this compensating motion vector, carries out steady picture to this current frame image and compensates.
Further, compensating unit 304, also for being the later image of the second two field picture when this current frame image, before this current frame image being carried out to steady picture compensation, determine that the compensating motion vector of the compensating motion vector of this current frame image and the previous frame image of this current frame image meets first and presets condition of similarity.
Further, compensating unit 304, also for the compensating motion vector of the previous frame image when the compensating motion vector and this current frame image of determining this current frame image do not meet this first preset condition of similarity time, this current frame image is divided into the image-region of at least the first predetermined number homalographic;
Contain each image-region in the image-region of this estimation of motion vectors point of at least the second predetermined number for the first predetermined number, adopt following steps to determine that quantity is unanimously put in the region of this image-region:
Determine the point motion vector of each estimation of motion vectors point in this image-region; For each designated movement vector estimation point in this image-region, determine the quantity of the consistent point of this designated movement vector estimation point in other each estimation of motion vectors point, wherein, designated movement vector estimation point is the estimation of motion vectors point of the second predetermined number chosen from all estimation of motion vectors points this image-region, and the consistent point of this designated movement vector estimation point is the estimation of motion vectors point that the point motion vector of point motion vector and this designated movement vector estimation point meets that second presets condition of similarity; Determine the maximum of each self-corresponding quantity that this is unanimously put of each designated movement vector estimation point, quantity is unanimously put in the region as this image-region;
When unanimously the maximum put in quantity is less than amount threshold in each this region self-corresponding of image-region of individual this estimation of motion vectors point containing at least the second predetermined number of this first predetermined number, cancels and steady picture compensation is carried out to this current frame image; When unanimously the maximum put in quantity is not less than amount threshold in each this region self-corresponding of image-region of individual this estimation of motion vectors point containing at least the second predetermined number of this first predetermined number, based on this compensating motion vector, steady picture is carried out to this current frame image and compensates.
Further, this steady picture device, also comprises:
Updating block 305, compensate for carrying out steady picture in cancellation to this current frame image, or this current frame image is carried out steady as after compensation, when the compensating motion vector of the previous frame image of the compensating motion vector and this current frame image of determining this current frame image does not meet this first default condition of similarity, upgrade the next frame image that this current reference frame image is this current frame image or this current frame image; Or carry out steady picture in cancellation to this current frame image to compensate, or this current frame image is carried out steady as after compensation, when this current reference frame image is greater than frequency threshold value as the number of times with reference to two field picture, upgrade the next frame image that this current reference frame image is this current frame image or this current frame image.
Further, the second determining unit 303, specifically for determining the compensating motion vector of this current frame image by random sampling unification algorism.
Further, the second determining unit 303, specifically for being divided into the image-region of at least the first predetermined number homalographic by this current frame image;
Contain each image-region in the image-region of this estimation of motion vectors point of at least the second predetermined number for the first predetermined number, adopt following steps to determine the consistent point motion vector in the region of this image-region:
Determine the point motion vector of each estimation of motion vectors point in this image-region; For each designated movement vector estimation point in this image-region, determine the quantity of the consistent point of this designated movement vector estimation point in other each estimation of motion vectors point, wherein, designated movement vector estimation point is the estimation of motion vectors point of the second predetermined number chosen from all estimation of motion vectors points this image-region, and the consistent point of this designated movement vector estimation point is the estimation of motion vectors point that the point motion vector of point motion vector and this designated movement vector estimation point meets that second presets condition of similarity; Determine the point motion vector of the designated movement vector estimation point that the maximum of each self-corresponding quantity that this is unanimously put of each designated movement vector estimation point is corresponding, the consistent point motion vector in the region as this image-region;
For often kind of vector in this first predetermined number consistent point motion vector in this region, following steps are adopted to determine the compensation vector that this kind of vector is corresponding:
For each appointment vector in this kind of vector, determine the quantity of the consistent vector of this appointment vector in other each vector, wherein, appointment vector is the vector of the 3rd predetermined number chosen from this first predetermined number this kind of vector, and the consistent vector of this appointment vector is meet the vector that the 3rd presets condition of similarity with this appointment vector; Determine the mean value of the consistent vector of the appointment vector that the maximum of the quantity of each this consistent vector self-corresponding of each appointment vector is corresponding, as the compensation vector that this kind of vector is corresponding;
Based on the compensation vector that various vector is corresponding, obtain the compensating motion vector of this current frame image.
Further, the first determining unit 302, specifically for being divided at least a 3rd predetermined number image block by this current frame image; Determine that this partial moving picture block is that the absolute difference of local horizontal motion vector and global level motion vector in a 3rd predetermined number image block is greater than differential horizontal threshold value, and/or the absolute difference of partial vertical motion vector and overall vertical motion vector is greater than the image block of differential vertical threshold value; This local horizontal motion vector is the horizontal motion vector of image block relative to the image block of same position in this current reference frame image of this current frame image; This global level motion vector is the horizontal motion vector of this current frame image relative to this current reference frame image; This partial vertical motion vector is the vertical motion vector of image block relative to the image block of same position in this current reference frame image of this current frame image; This overall vertical motion vector is the vertical motion vector of this current frame image relative to this current reference frame image.
The function of above-mentioned each module may correspond to the respective handling step in flow process shown in Fig. 1 or Fig. 2, does not repeat them here.
In sum, the scheme that the embodiment of the present invention provides, determines that from the characteristic point of current frame image the specific characteristic point with the characteristic point characteristic matching of current reference two field picture, this characteristic point are meet the point presetting characteristic condition in image; Determine that the estimation of motion vectors point of this current frame image is in all specific characteristics point, other specific characteristic point except the specific characteristic point in partial moving picture block; This partial moving picture block is the image block comprising Moving Objects image in this current frame image; Based on the coordinate of this estimation of motion vectors point in this current frame image, and in this current reference frame image with the coordinate of the characteristic point of this estimation of motion vectors points correspondence, determine the compensating motion vector of this current frame image; Based on this compensating motion vector, steady picture is carried out to this current frame image and compensates.Adopt scheme provided by the invention, solve the problem of image steady picture weak effect.
What the embodiment of the application provided surely realizes by computer program as device.Those skilled in the art should be understood that; above-mentioned Module Division mode is only the one in numerous Module Division mode; if be divided into other modules or do not divide module, as long as surely there is above-mentioned functions as device, all should within the protection range of the application.
The application describes with reference to according to the flow chart of the method for the embodiment of the present application, equipment (system) and computer program and/or block diagram.Should understand can by the combination of the flow process in each flow process in computer program instructions realization flow figure and/or block diagram and/or square frame and flow chart and/or block diagram and/or square frame.These computer program instructions can being provided to the processor of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing device to produce a machine, making the instruction performed by the processor of computer or other programmable data processing device produce device for realizing the function of specifying in flow chart flow process or multiple flow process and/or block diagram square frame or multiple square frame.
These computer program instructions also can be stored in can in the computer-readable memory that works in a specific way of vectoring computer or other programmable data processing device, the instruction making to be stored in this computer-readable memory produces the manufacture comprising command device, and this command device realizes the function of specifying in flow chart flow process or multiple flow process and/or block diagram square frame or multiple square frame.
These computer program instructions also can be loaded in computer or other programmable data processing device, make on computer or other programmable devices, to perform sequence of operations step to produce computer implemented process, thus the instruction performed on computer or other programmable devices is provided for the step realizing the function of specifying in flow chart flow process or multiple flow process and/or block diagram square frame or multiple square frame.
Obviously, those skilled in the art can carry out various change and modification to the present invention and not depart from the spirit and scope of the present invention.Like this, if these amendments of the present invention and modification belong within the scope of the claims in the present invention and equivalent technologies thereof, then the present invention is also intended to comprise these change and modification.

Claims (12)

1. a digital image stabilization method, is characterized in that, comprising:
From the characteristic point of current frame image, determine that the specific characteristic point with the characteristic point characteristic matching of current reference two field picture, described characteristic point are meet the point presetting characteristic condition in image;
Determine that the estimation of motion vectors point of described current frame image is in all specific characteristics point, other specific characteristic point except the specific characteristic point in partial moving picture block; Described partial moving picture block is the image block comprising Moving Objects image in described current frame image;
Based on the coordinate of the described estimation of motion vectors point in described current frame image, and in described current reference two field picture with the coordinate of the characteristic point of described estimation of motion vectors points correspondence, determine the compensating motion vector of described current frame image;
Based on described compensating motion vector, steady picture is carried out to described current frame image and compensates;
Wherein, when described current frame image is the later image of the second two field picture, carrying out described current frame image before steady picture compensates, also comprising:
Determine that the compensating motion vector of the compensating motion vector of described current frame image and the previous frame image of described current frame image meets first and presets condition of similarity.
2. the method for claim 1, is characterized in that, also comprises:
When the compensating motion vector of the previous frame image of the compensating motion vector and described current frame image of determining described current frame image does not meet the described first default condition of similarity, described current frame image is divided into the image-region of at least the first predetermined number homalographic;
Contain each image-region in the image-region of the described estimation of motion vectors point of at least the second predetermined number for the first predetermined number, adopt following steps to determine that quantity is unanimously put in the region of this image-region:
Determine the point motion vector of each estimation of motion vectors point in this image-region;
For each designated movement vector estimation point in this image-region, determine the quantity of the consistent point of this designated movement vector estimation point in other each estimation of motion vectors point, wherein, designated movement vector estimation point is the estimation of motion vectors point of the second predetermined number chosen from all estimation of motion vectors points this image-region, and the consistent point of this designated movement vector estimation point is the estimation of motion vectors point that the point motion vector of point motion vector and this designated movement vector estimation point meets that second presets condition of similarity;
Determine the maximum of the quantity of each self-corresponding described consistent point of each designated movement vector estimation point, quantity is unanimously put in the region as this image-region;
When unanimously the maximum put in quantity is less than amount threshold in each self-corresponding described region of image-region of the individual described estimation of motion vectors point containing at least the second predetermined number of described first predetermined number, cancels and steady picture compensation is carried out to described current frame image;
When unanimously the maximum put in quantity is not less than amount threshold in each self-corresponding described region of image-region of the individual described estimation of motion vectors point containing at least the second predetermined number of described first predetermined number, based on described compensating motion vector, steady picture is carried out to described current frame image and compensates.
3. method as claimed in claim 2, is characterized in that, carries out surely picture compensate in cancellation to described current frame image, or after carrying out steady picture compensation to described current frame image, also comprises:
When the compensating motion vector of the previous frame image of the compensating motion vector and described current frame image of determining described current frame image does not meet the described first default condition of similarity, upgrade the next frame image that described current reference two field picture is described current frame image or described current frame image; Or
When described current reference two field picture is greater than frequency threshold value as the number of times with reference to two field picture, upgrade the next frame image that described current reference two field picture is described current frame image or described current frame image.
4. the method for claim 1, is characterized in that, determines the compensating motion vector of described current frame image, is specially:
The compensating motion vector of described current frame image is determined by random sampling unification algorism.
5. method as claimed in claim 4, be is characterized in that, determined the compensating motion vector of described current frame image, be specially by random sampling unification algorism:
Described current frame image is divided into the image-region of at least the first predetermined number homalographic;
Contain each image-region in the image-region of the described estimation of motion vectors point of at least the second predetermined number for the first predetermined number, adopt following steps to determine the consistent point motion vector in the region of this image-region:
Determine the point motion vector of each estimation of motion vectors point in this image-region;
For each designated movement vector estimation point in this image-region, determine the quantity of the consistent point of this designated movement vector estimation point in other each estimation of motion vectors point, wherein, designated movement vector estimation point is the estimation of motion vectors point of the second predetermined number chosen from all estimation of motion vectors points this image-region, and the consistent point of this designated movement vector estimation point is the estimation of motion vectors point that the point motion vector of point motion vector and this designated movement vector estimation point meets that second presets condition of similarity;
Determine the point motion vector of the designated movement vector estimation point that the maximum of the quantity of each self-corresponding described consistent point of each designated movement vector estimation point is corresponding, the consistent point motion vector in the region as this image-region;
For often kind of vector in a described first predetermined number consistent point motion vector in described region, following steps are adopted to determine the compensation vector that this kind of vector is corresponding:
For each appointment vector in this kind of vector, determine the quantity of the consistent vector of this appointment vector in other each vector, wherein, appointment vector is the vector of the 3rd predetermined number chosen from described first predetermined number this kind of vector, and the consistent vector of this appointment vector is meet the vector that the 3rd presets condition of similarity with this appointment vector;
Determine the mean value of the consistent vector of the appointment vector that the maximum of the quantity of each self-corresponding described consistent vector of each appointment vector is corresponding, as the compensation vector that this kind of vector is corresponding;
Based on the compensation vector that various vector is corresponding, obtain the compensating motion vector of described current frame image.
6. the method for claim 1, is characterized in that, specifically determines described partial moving picture block in the following way:
Described current frame image is divided at least a 3rd predetermined number image block; Determine that described partial moving picture block is that the absolute difference of local horizontal motion vector and global level motion vector in a 3rd predetermined number image block is greater than differential horizontal threshold value, and/or the absolute difference of partial vertical motion vector and overall vertical motion vector is greater than the image block of differential vertical threshold value; Described local horizontal motion vector is the horizontal motion vector of image block relative to the image block of same position in described current reference two field picture of described current frame image; Described global level motion vector is the horizontal motion vector of described current frame image relative to described current reference two field picture; Described partial vertical motion vector is the vertical motion vector of image block relative to the image block of same position in described current reference two field picture of described current frame image; Described overall vertical motion vector is the vertical motion vector of described current frame image relative to described current reference two field picture.
7. a steady picture device, is characterized in that, comprising:
Matching unit, for determining the specific characteristic point with the characteristic point characteristic matching of current reference two field picture from the characteristic point of current frame image, described characteristic point is meet the point presetting characteristic condition in image;
First determining unit, for determining that the estimation of motion vectors point of described current frame image is in all specific characteristics point, other specific characteristic point except the specific characteristic point in partial moving picture block; Described partial moving picture block is the image block comprising Moving Objects image in described current frame image;
Second determining unit, for the coordinate based on the described estimation of motion vectors point in described current frame image, with in described current reference two field picture with the coordinate of the characteristic point of described estimation of motion vectors points correspondence, determine the compensating motion vector of described current frame image;
Compensating unit, for based on described compensating motion vector, carries out steady picture to described current frame image and compensates;
Wherein, described compensating unit, also for being the later image of the second two field picture when described current frame image, before described current frame image being carried out to steady picture compensation, determine that the compensating motion vector of the compensating motion vector of described current frame image and the previous frame image of described current frame image meets first and presets condition of similarity.
8. device as claimed in claim 7, it is characterized in that, described compensating unit, also for the compensating motion vector of the previous frame image when the compensating motion vector and described current frame image of determining described current frame image do not meet described first preset condition of similarity time, described current frame image is divided into the image-region of at least the first predetermined number homalographic;
Contain each image-region in the image-region of the described estimation of motion vectors point of at least the second predetermined number for the first predetermined number, adopt following steps to determine that quantity is unanimously put in the region of this image-region:
Determine the point motion vector of each estimation of motion vectors point in this image-region; For each designated movement vector estimation point in this image-region, determine the quantity of the consistent point of this designated movement vector estimation point in other each estimation of motion vectors point, wherein, designated movement vector estimation point is the estimation of motion vectors point of the second predetermined number chosen from all estimation of motion vectors points this image-region, and the consistent point of this designated movement vector estimation point is the estimation of motion vectors point that the point motion vector of point motion vector and this designated movement vector estimation point meets that second presets condition of similarity; Determine the maximum of the quantity of each self-corresponding described consistent point of each designated movement vector estimation point, quantity is unanimously put in the region as this image-region;
When unanimously the maximum put in quantity is less than amount threshold in each self-corresponding described region of image-region of the individual described estimation of motion vectors point containing at least the second predetermined number of described first predetermined number, cancels and steady picture compensation is carried out to described current frame image; When unanimously the maximum put in quantity is not less than amount threshold in each self-corresponding described region of image-region of the individual described estimation of motion vectors point containing at least the second predetermined number of described first predetermined number, based on described compensating motion vector, steady picture is carried out to described current frame image and compensates.
9. device as claimed in claim 8, is characterized in that, also comprise:
Updating block, compensate for carrying out steady picture in cancellation to described current frame image, or described current frame image is carried out steady as after compensation, when the compensating motion vector of the previous frame image of the compensating motion vector and described current frame image of determining described current frame image does not meet the described first default condition of similarity, upgrade the next frame image that described current reference two field picture is described current frame image or described current frame image; Or carry out steady picture in cancellation to described current frame image to compensate, or described current frame image is carried out steady as after compensation, when described current reference two field picture is greater than frequency threshold value as the number of times with reference to two field picture, upgrade the next frame image that described current reference two field picture is described current frame image or described current frame image.
10. device as claimed in claim 7, be is characterized in that, described second determining unit, specifically for being determined the compensating motion vector of described current frame image by random sampling unification algorism.
11. devices as claimed in claim 10, is characterized in that, described second determining unit, specifically for described current frame image being divided into the image-region of at least the first predetermined number homalographic;
Contain each image-region in the image-region of the described estimation of motion vectors point of at least the second predetermined number for the first predetermined number, adopt following steps to determine the consistent point motion vector in the region of this image-region:
Determine the point motion vector of each estimation of motion vectors point in this image-region; For each designated movement vector estimation point in this image-region, determine the quantity of the consistent point of this designated movement vector estimation point in other each estimation of motion vectors point, wherein, designated movement vector estimation point is the estimation of motion vectors point of the second predetermined number chosen from all estimation of motion vectors points this image-region, and the consistent point of this designated movement vector estimation point is the estimation of motion vectors point that the point motion vector of point motion vector and this designated movement vector estimation point meets that second presets condition of similarity; Determine the point motion vector of the designated movement vector estimation point that the maximum of the quantity of each self-corresponding described consistent point of each designated movement vector estimation point is corresponding, the consistent point motion vector in the region as this image-region;
For often kind of vector in a described first predetermined number consistent point motion vector in described region, following steps are adopted to determine the compensation vector that this kind of vector is corresponding:
For each appointment vector in this kind of vector, determine the quantity of the consistent vector of this appointment vector in other each vector, wherein, appointment vector is the vector of the 3rd predetermined number chosen from described first predetermined number this kind of vector, and the consistent vector of this appointment vector is meet the vector that the 3rd presets condition of similarity with this appointment vector; Determine the mean value of the consistent vector of the appointment vector that the maximum of the quantity of each self-corresponding described consistent vector of each appointment vector is corresponding, as the compensation vector that this kind of vector is corresponding;
Based on the compensation vector that various vector is corresponding, obtain the compensating motion vector of described current frame image.
12. devices as claimed in claim 7, is characterized in that, described first determining unit, specifically for described current frame image being divided at least a 3rd predetermined number image block; Determine that described partial moving picture block is that the absolute difference of local horizontal motion vector and global level motion vector in a 3rd predetermined number image block is greater than differential horizontal threshold value, and/or the absolute difference of partial vertical motion vector and overall vertical motion vector is greater than the image block of differential vertical threshold value; Described local horizontal motion vector is the horizontal motion vector of image block relative to the image block of same position in described current reference two field picture of described current frame image; Described global level motion vector is the horizontal motion vector of described current frame image relative to described current reference two field picture; Described partial vertical motion vector is the vertical motion vector of image block relative to the image block of same position in described current reference two field picture of described current frame image; Described overall vertical motion vector is the vertical motion vector of described current frame image relative to described current reference two field picture.
CN201210463920.8A 2012-11-15 2012-11-15 A kind of digital image stabilization method and device Active CN103813056B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201210463920.8A CN103813056B (en) 2012-11-15 2012-11-15 A kind of digital image stabilization method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201210463920.8A CN103813056B (en) 2012-11-15 2012-11-15 A kind of digital image stabilization method and device

Publications (2)

Publication Number Publication Date
CN103813056A CN103813056A (en) 2014-05-21
CN103813056B true CN103813056B (en) 2016-03-16

Family

ID=50709221

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201210463920.8A Active CN103813056B (en) 2012-11-15 2012-11-15 A kind of digital image stabilization method and device

Country Status (1)

Country Link
CN (1) CN103813056B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108805908A (en) * 2018-06-08 2018-11-13 浙江大学 A kind of real time video image stabilization based on the superposition of sequential grid stream

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105007397B (en) * 2015-04-30 2018-06-19 南方电网科学研究院有限责任公司 Video compensation method for eliminating line segment cross mismatching points
CN107911615B (en) * 2017-12-26 2019-07-02 北京理工大学 A kind of Gray Projection digital image stabilization method and system
CN108111760B (en) * 2017-12-26 2019-09-10 北京理工大学 A kind of electronic image stabilization method and system
CN109003245B (en) * 2018-08-21 2021-06-04 厦门美图之家科技有限公司 Coordinate processing method and device and electronic equipment
CN110401796B (en) * 2019-07-05 2020-09-29 浙江大华技术股份有限公司 Jitter compensation method and device of image acquisition device
CN110602393B (en) * 2019-09-04 2020-06-05 南京博润智能科技有限公司 Video anti-shake method based on image content understanding
CN115442522B (en) * 2022-08-10 2023-11-21 深圳市贝嘉技术有限公司 Imaging adjustment method, device, equipment and storage medium based on microscope

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101511024A (en) * 2009-04-01 2009-08-19 北京航空航天大学 Movement compensation method of real time electronic steady image based on motion state recognition
CN102148934A (en) * 2011-04-02 2011-08-10 北京理工大学 Multi-mode real-time electronic image stabilizing system
CN102231792A (en) * 2011-06-29 2011-11-02 南京大学 Electronic image stabilization method based on characteristic coupling
CN102780846A (en) * 2012-07-11 2012-11-14 清华大学 Electronic image stabilization method based on inertial navigation information

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101511024A (en) * 2009-04-01 2009-08-19 北京航空航天大学 Movement compensation method of real time electronic steady image based on motion state recognition
CN102148934A (en) * 2011-04-02 2011-08-10 北京理工大学 Multi-mode real-time electronic image stabilizing system
CN102231792A (en) * 2011-06-29 2011-11-02 南京大学 Electronic image stabilization method based on characteristic coupling
CN102780846A (en) * 2012-07-11 2012-11-14 清华大学 Electronic image stabilization method based on inertial navigation information

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108805908A (en) * 2018-06-08 2018-11-13 浙江大学 A kind of real time video image stabilization based on the superposition of sequential grid stream
CN108805908B (en) * 2018-06-08 2020-11-03 浙江大学 Real-time video image stabilization method based on time sequence grid stream superposition

Also Published As

Publication number Publication date
CN103813056A (en) 2014-05-21

Similar Documents

Publication Publication Date Title
CN103813056B (en) A kind of digital image stabilization method and device
CN107925755B (en) Method and system for planar surface detection for image processing
US9973696B1 (en) Apparatus and methods for image alignment
US20130322766A1 (en) Method of detecting global motion and global motion detector, and digital image stabilization (dis) method and circuit including the same
US9792709B1 (en) Apparatus and methods for image alignment
US10957093B2 (en) Scene-based foveated rendering of graphics content
US9161015B2 (en) Image processing apparatus and method, and program
US10489897B2 (en) Apparatus and methods for artifact detection and removal using frame interpolation techniques
US20150103193A1 (en) Method and apparatus for long term image exposure with image stabilization on a mobile device
CN111062981B (en) Image processing method, device and storage medium
US20170269187A1 (en) Image pickup apparatus having function of generating simulation image,control method therefor, and storage medium
DE102020128899A1 (en) ADAPTIVE VIRTUAL CAMERA FOR INDIRECT-SPARSE-SIMULTANEOUS-LOCALIZATION AND MAPPING SYSTEMS
TW201319954A (en) Image stabilization method and image stabilization device
US20190362505A1 (en) Image processing apparatus, method, and storage medium to derive optical flow
CN103402045A (en) Image de-spin and stabilization method based on subarea matching and affine model
CN104299188B (en) Image correcting method and system
CN112308797B (en) Corner detection method and device, electronic equipment and readable storage medium
CN108596946A (en) A kind of moving target real-time detection method and system
CN114125411B (en) Projection device correction method, projection device correction device, storage medium and projection device
CN115701125A (en) Image anti-shake method and electronic equipment
KR101806453B1 (en) Moving object detecting apparatus for unmanned aerial vehicle collision avoidance and method thereof
KR20200096426A (en) Moving body detecting device, moving body detecting method, and moving body detecting program
US20230290061A1 (en) Efficient texture mapping of a 3-d mesh
DE102022111926A1 (en) Method and apparatus for depth based inertial visual dometry
US11847784B2 (en) Image processing apparatus, head-mounted display, and method for acquiring space information

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant