CN100389601C - Video electronic flutter-proof device - Google Patents
Video electronic flutter-proof device Download PDFInfo
- Publication number
- CN100389601C CN100389601C CNB2005101080362A CN200510108036A CN100389601C CN 100389601 C CN100389601 C CN 100389601C CN B2005101080362 A CNB2005101080362 A CN B2005101080362A CN 200510108036 A CN200510108036 A CN 200510108036A CN 100389601 C CN100389601 C CN 100389601C
- Authority
- CN
- China
- Prior art keywords
- relative motion
- motion vectors
- present frame
- overall relative
- motion vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 239000013598 vector Substances 0.000 claims abstract description 215
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 4
- 238000000034 method Methods 0.000 claims description 32
- 238000006073 displacement reaction Methods 0.000 claims description 10
- 230000008676 import Effects 0.000 claims description 4
- 238000004513 sizing Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 206010034972 Photosensitivity reaction Diseases 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000036211 photosensitivity Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
Images
Landscapes
- Studio Devices (AREA)
Abstract
The present invention discloses a video electronic flutter-proof device having the advantages of simplicity, effectiveness and cost saving, which comprises a motion estimation computation module, a global relative motion vector computation module, a global relative motion vector storing module, a computation module of motion vector for compensating flutter and a flutter compensation module, wherein the motion estimation computation module is used for computing relative motion vector in a current frame inner search block by a motion estimation algorithm; the global relative motion vector computation module is used for obtaining the global relative motion vector of a current frame; the global relative motion vector storing module is used for storing the global relative motion vector; the computation module of motion vector for compensating flutter is used for predicting the global relative motion vector of the current frame and estimating the difference of global corresponding motion vector and the predicted global relative motion vector of the current frame by motion to judge whether the flutter exists and determine and output the motion vector for compensating flutter; the flutter compensation module is used for inputting a total pixel image of the current frame and cutting an effective image after the current frame is in flutter-proof compensation from the total pixel image of the current frame according to the motion vector for compensating flutter and the size of a preestablished effective pixel image.
Description
Technical field
The present invention relates to a kind of video electronic flutter-proof device that is applied in video acquisition and the output equipment.
Technical background
Along with the progress and the development of science and technology, equipment such as digital camera, Digital Video have increased many new functions.But these handheld devices inevitably cause the shake of image owing to artificial factor.The avoiding and to realize by anti-shake technology of this shake.The anti-shake system of present stage mainly is divided into two big classes: the anti-shake and electronic flutter-proof of optics.Optics is anti-shake to be by the shake of the built-in instrument induction camera of camera lens, reaches anti-shake effect by the position of adjusting lens in the camera lens again.The anti-shake technology of optics is generally used in the high-end digital camera and Digital Video.Electronic flutter-proof is to come image is handled by electronically, to alleviate the influence of shake to imaging.At present, electronic flutter-proof mainly contains three kinds of enforcement means, improves ISO photosensitivity/electronic image compression/BSS pattern automatically.The anti-shake principle of CCD is exactly that CCD is placed on the support that can move up and down, and detects earlier whether shake is arranged, owing to use gyrosensor, the detection of shake and other companies are basic identical.Sensor goes out direction, speed, the amount of movement of shake then ... the signal that detects calculates the CCD amount of movement that can be enough to offset shake through handling.Electronic flutter-proof is generally used in the digital camera and Digital Video of low side owing to cheap.
This shows that optics is anti-shake respectively to have pluses and minuses with two kinds of technology of electronic flutter-proof.The anti-shake technology of optics anti-shake effective can make full use of the collection pixel of CCD, but need the lens movement compensation arrangement, so cost is higher relatively.Electronic flutter-proof need not the lens movement compensation arrangement, and is cheap, realizes simply, still reduces the utilance of CCD; Adopt the CCD compensation method then to need extra CCD mobile device, also to a certain degree improved cost.And no matter be the anti-shake or electronic flutter-proof of optics, generally all need a motion transducer---the jitter compensation gyroscope comes the direction and the intensity of perceive motion, adopt optical means and electronic method to compensate again.
Summary of the invention
The objective of the invention is: at the deficiencies in the prior art, provide a kind of simple and reasonable, need not external motion sensor and the device of the video electronic flutter-proof that can reduce cost.
In order to solve the problems of the technologies described above, the technical solution used in the present invention is: a kind of device of video electronic flutter-proof, the motion vector computation module and the jitter compensation module that comprise estimation computing module, overall relative motion vectors computing module, overall relative motion vectors memory module, compensate for jitter, wherein
Described estimation computing module is imported the image of present frame and reference frame, and calculates the relative motion vectors of the relative reference frame of search block in the present frame according to motion estimation algorithm;
Described overall relative motion vectors computing module is imported the relative motion vectors from the search block of the present frame of described estimation computing module, thereby obtains the overall relative motion vectors of present frame according to the relative motion vectors of each search block;
Described overall relative motion vectors memory module is used to store the overall relative motion vectors of present frame some two field pictures before;
The motion vector computation module of described compensate for jitter is according to the overall relative motion vectors of predicting present frame from the overall relative motion vectors of some frames before the present frame of storing in the overall relative motion vectors memory module, and quite motion vector and the motion vector computation module of described compensate for jitter are predicted the difference of the overall relative motion vectors of the present frame that obtains to calculate the overall situation of present frame of overall relative motion vectors computing module output, thereby judge whether to exist shake according to this difference, when having shake, the motion vector of shaking by way of compensation with described difference, when not having shake, the motion vector that makes compensate for jitter is 0; And whether the motion vector of judging described compensate for jitter is in the determined maximum compensation range of predefined maximum compensating motion vector value, if exceed this scope, then with the motion vector of described maximum compensating motion vector value as described compensate for jitter; The last motion vector of compensate for jitter of exporting again is to described jitter compensation module;
Total pixel image of described jitter compensation module input present frame, and according to the motion vector and the predefined valid pixel image size of described compensate for jitter, cutting obtains the effective image after the anti-shake compensation of present frame from total pixel image of present frame.
Further, can also comprise the pre-cut module, be used to import the overall relative motion vectors of each frame before total pixel image of present frame and the present frame, thereby obtain the absolute displacement value of the former frame image of present frame with respect to total pixel image center according to the overall relative motion vectors of each frame; And with this absolute displacement value definite center, clipping region on total pixel image of present frame, again according to the image size of predefined valid pixel, determine the clipping region with the center, clipping region, total pixel image to present frame carries out cutting at last, thereby the output size is the current frame video image of described valid pixel image size.
Further, can also comprise overall relative motion vectors correcting module, be used to import the overall relative motion vectors of the present frame of the motion vector of compensate for jitter of present frame and estimation gained, thereby, and export described overall relative motion vectors memory module to and store with the motion vector of described overall relative motion vectors and compensate for jitter overall relative motion vectors as revised present frame.
Further, also can comprise the reference frame storing module, be used for effective image of the present frame after the input jiffer compensation, reference frame when calculating as the next frame estimation.
Preferably, after described overall relative motion vectors computing module is imported the relative motion vectors of each search block, thereby the frequency of the relative motion vectors amplitude of described each search block of statistics obtains the relative motion vectors value that wherein frequency is the highest, and calculates the variance of described relative motion vectors; Judge then whether the highest relative motion vectors value of described frequency has surpassed the variance of the relative motion vectors of search block and the product of given threshold value in the image, if do not surpass, the overall relative motion vectors that then shows current frame image is not obvious, and the overall relative motion vectors of putting present frame is 0; Otherwise, with the overall relative motion vectors of the highest relative motion vectors value of described frequency as present frame.
Preferably, the motion vector computation module of described compensate for jitter can adopt the overall relative motion vectors of the method prediction present frame of curve fit, is specifically as follows:
At first, according to the overall relative motion vectors of preceding some frames of present frame, determine a multinomial of p time:
P(x)＝w
px
p+w
p-1x
p-1+...+w
0
Wherein, x is relative frame number, and P (x) is that relative frame number is the overall relative motion vectors of the picture frame of x, and p is determined by the global motion pattern for describing the exponent number of overall relative motion vectors track;
Then, adopt described polynomial computation to obtain the overall relative motion vectors of present frame, be the overall relative motion vectors that forecasting institute gets.
Preferably, the motion vector computation module of described compensate for jitter judges whether to exist the method for shake specifically to can be according to the difference of the overall relative motion vectors of estimation and two kinds of method gained of prediction: at first, and the variance of the overall relative motion vectors of preceding some frames of calculating present frame; If: | the overall relative motion vectors of the present frame that overall relative motion vectors one estimation of the present frame that prediction obtains obtains | the variance of the overall relative motion vectors of preceding some frames of＞K* present frame, represent that then there is shake in present frame, otherwise think not have shake, wherein 1≤K≤2.
Preferably, described jitter compensation module is on total pixel image of present frame, and according to the motion vector and the predefined valid pixel image size of compensate for jitter, the step of the effective image after the anti-shake compensation of cutting acquisition present frame specifically can comprise:
At first, be central point with the center of total pixel image of present frame, obtain the compensation central point according to the motion vector of compensate for jitter;
Then, be the center with described compensation central point, according to described valid pixel image size, on total pixel image of present frame, determine the clipping region;
At last, exporting image pixel in the described clipping region is effective image after the anti-shake compensation.
Preferably, processing to overall relative motion vectors in described each module all can be carried out from horizontal direction and vertical direction respectively, thereby the overall relative motion vectors of the final present frame that obtains is horizontal component and the vertical component by overall relative motion vectors to be constituted, the motion vector of the jitter compensation that is obtained also is made of its horizontal component and vertical component, and then to obtain effective image also be to carry out from horizontal direction and vertical direction respectively to the judgement of shake and cutting.
Preferably, described search block can be all macro blocks or the piece in the two field picture.
Preferably, described search block can be the data block of the pre-sizing in the two field picture, and described data block keeps at a certain distance away each other.
In technique scheme, a kind of video electronic flutter-proof device proposed by the invention is by image sequence being carried out estimation, analyze the motion conditions of photographed, eliminating shake thereby carry out motion compensation.The relative prior art of the present invention, need not external motion sensor, realize the estimation of motion, the compensation of dither image by digital image processing method fully, and have realize simple, effectively, the characteristics of low cost etc., can be applied in Digital Video, the first-class video acquisition device of shooting.
Description of drawings
Accompanying drawing 1 carries out the schematic diagram of video electronic flutter-proof for the present invention;
Accompanying drawing 2 carries out the anti-shake compensation front and back image schematic diagram of video electronic flutter-proof for the present invention;
Accompanying drawing 3 carries out the flow chart of video electronic flutter-proof for the present invention;
Accompanying drawing 4 is the circuit block diagram of a kind of video electronic flutter-proof device of the present invention;
Accompanying drawing 5 three two field pictures after for cutting among the present invention concern schematic diagram with respect to the position of total pixel image;
Accompanying drawing 6 is the schematic diagram of a kind of preferable estimation method of the present invention;
Accompanying drawing 7 is the schematic diagram of the anti-shake example of a reality of the present invention.
Embodiment
Below in conjunction with Figure of description and specific embodiment the present invention is described in further detail.
The electronic flutter-proof method that present embodiment proposes need not to add the motion compensation transducer and obtains motion state, but introduced the method for estimating of video coding technique, calculate the overall relative motion vectors of every frame by method for estimating, thereby obtain the overall relative motion vectors MV of present frame
NEstimated value, and the overall relative motion vectors MV by previous some frames
1, MV
2..., MV
N-1The overall relative motion vectors MV of prediction present frame
NIf MV
NAnd MV
NBetween the difference that exists greater than a certain given threshold value T, then think shake taken place, need carry out motion compensation; Otherwise keep original image and do not compensate.Its principle as shown in Figure 1.
Present embodiment can not utilize the collection pixel value of the CCD of all videos harvester in order to realize electronic flutter-proof, and therefore effectively pixel count is less than the total pixel number of CCD, and this is the situation that realizes that electronic flutter-proof must occur.And the ratio of the relative total pixel number of valid pixel number is related by the anti-shake jitter amplitude of supporting.Suppose maximum jitter that present embodiment supports by relative shifting range as index, the effective pixel area of given CCD is the center of the total pixel region of CCD, then the compensation range of Zui Da shake can not exceed the zone of the total pixel of CCD, therefore preestablishing the maximum compensating motion vector value of going up in the horizontal direction in the present embodiment is XMAX, maximum compensating motion vector value on the vertical direction is YMAX, then maximum compensation range in the horizontal direction is [XMAX, XMAX], maximum compensation range in vertical direction is [YMAX, YMAX].If the shake total size is positioned at this scope, then can compensate the realization electronic flutter-proof; If surpass this scope, then do not carry out electronic flutter-proof.The total pixel of CCD, the relation of the valid pixel after valid pixel and the anti-shake compensation as shown in Figure 2.
With reference to figure 3, the idiographic flow that present embodiment carries out video electronic flutter-proof is:
The overall relative motion vectors of some frame video images before step 1, employing method for estimating calculating present frame and the present frame;
The difference of the overall relative motion vectors of the present frame that obtains in step 3, calculation procedure 1 and the step 2, and judge whether to exist shake according to this difference, and when having shake, the motion vector of shaking by way of compensation with described difference, when not having shake, the motion vector that makes compensate for jitter is 0;
Step 4, judge that the motion vector of described compensate for jitter is whether in the determined maximum compensation range of predefined maximum compensating motion vector value, if exceed this scope, then with the motion vector of described maximum compensating motion vector value as described compensate for jitter;
In above-mentioned steps, the order of step 1 and step 2 can be changed.For the overall relative motion vectors that makes present frame estimate with prediction accurately but amount of calculation is relative less, adopt the F two field picture before the present frame in the present embodiment.
In above-mentioned steps, the video image that is used to carry out estimation can be total pixel image, also can be effective image of the valid pixel size after the compensate for jitter.Adopt effective image in the present embodiment.
Based on above-mentioned flow process, present embodiment provides a kind of device of video electronic flutter-proof, comprise the motion vector computation module and the jitter compensation module of estimation computing module, overall relative motion vectors computing module, overall relative motion vectors memory module, compensate for jitter, also comprise pre-cut module, overall relative motion vectors correcting module and reference frame storing module in addition.
Below in conjunction with the device of present embodiment shown in the accompanying drawing 4, describe the principle of present embodiment device in detail.
(1) pre-cut module
This module is input as the overall relative motion vectors of each frame before total pixel image of the present frame that the video acquisition module (as CCD) of video acquisition and output device gathers and the selected present frame, is output as the image of the image size (being the valid pixel size) of actual displayed.
Suppose that total pixel image size that CCD gathers is M * N, the image size of valid pixel is M ' * N '.We use MV
X(i) and MV
Y(i) expression i two field picture is with respect to the relative motion vectors of i-1 two field picture.When i=1, the valid pixel image is positioned at the center of total pixel image, first two field picture in the expression video sequence, and at this moment initial value is respectively MV
X(1)=0, MV
Y(1)=0.We also need the absolute displacement (the just relative motion vectors of 1st frame) of i frame with respect to total pixel center position, use SMV respectively
X(i) and SMV
Y(i) expression.This value is obtained by following formula:
Absolute displacement has important effect in the anti-shake computing of every two field picture, absolute displacement can not surmount the restriction relation between total pixel and the valid pixel image, can be expressed as:
-XMAX≤SMV
X(i)≤XMAX
-YMAX≤SMV
Y(i)≤YMAX
When input i frame M * N sized images, according to the SMV of i-1
X(i) and SMV
Y(i) on total pixel image of present frame, determine the center, clipping region, according to the image size of predefined valid pixel current i two field picture is carried out cutting then, form the image of M ' * N '.For first frame data because SMV
XAnd SMV (1)=0
Y(1)=0, therefore the zone of actual cutting is the image of the M ' * N ' of the center of image.Figure 5 shows that three two field pictures after the cutting concern schematic diagram with respect to the position of total pixel image.Wherein, image A is total pixel image size that CCD gathers, and its center is the o point; Image B is the 1st two field picture after the cutting, and its center overlaps with the center of image A; Image C is (i-1) two field picture after the cutting, and its center is o ', and absolute the relative displacement of the 1st frame (just with respect to) displacement of its relative image B is SMV
x(i-1) and SMV
y(i-1); Image D is i frame (the being present frame) image after the cutting, and its center is o ", the absolute displacement of its relative image B is SMV
xI and SMV
yI, the relative motion vectors of its relative image C is MV
X(i) and MV
Y(i).
(2) estimation computing module
The estimation computing module is input as the i two field picture of the M ' * N ' after the cutting and i-1 two field picture as the reference two field picture.Be output as the relative motion vectors of i two field picture with respect to the i-1 two field picture.
Method for estimating can adopt based on the method for estimating in the video-frequency compression method, is that search block is carried out estimation with piece and macro block.At this moment can obtain the relative motion vectors of each macro block in the image or piece by estimation, as the input of overall relative motion vectors computing module.
Present embodiment recommend to adopt MPEG and H.26X waits motion estimation module in the video encoding standard to be applied directly to the motion estimation module of present embodiment.These methods can be easy to realize that directly use, shortcoming is that the method for estimating operand in the common video encoder is bigger.
Adopt a kind of method for estimating of simplification in the present embodiment, this method need not that the data block of all images is carried out estimation can significantly reduce operand, improves actual application value.
With reference to the accompanying drawings shown in 6, in this method for estimating, search block does not adopt the macro block described in the video encoder and the mode of piece, search block in this method is can be self-defining, usually select 8 * 8, the data block of 16 * 16 sizes, and the selection of search block is not entire image, but with certain partial data piece at interval.Search for the best match position in given zone of these data blocks.Be motion vector with the relative position of reference position.The motion match method still can adopt searching algorithm and matching criterior commonly used in the video encoder, for known technology is not described further here.
(3) overall relative motion vectors computing module
What this module was imported is the relative motion vectors of a plurality of search block of motion estimation module acquisition, is output as the overall relative motion vectors of the present frame that obtains by method for estimating.
The result who supposes the motion estimation module search is a L relative motion vectors.Add up the frequency of level and vertical motion vector amplitude respectively, suppose that the highest relative motion vectors value of frequency is respectively mvx
F, mvy
FRelative motion vectors is carried out statistical analysis, calculate variance and be respectively D
xAnd D
y
The principle that obtains overall relative motion vectors by method for estimating in the present embodiment is: judge whether current overall relative motion has surpassed the variance of the relative motion vectors of piece and the product of given threshold value in the image, if variance ratio is bigger, motion state in the key diagram picture is comparatively complicated, and the consistent relatively motion of the overall situation is not obvious, otherwise, the motion that the then explanation overall situation is consistent relatively is obvious, has overall relative motion vectors.
Be embodied as:
If mvx
F＜D
xT
x, show that the horizontal direction overall situation relative motion vectors of present image is not obvious, the horizontal component MVX (i)=0 of overall relative motion vectors is arranged; Otherwise show horizontal component MVX (the i)=mvx that has overall relative motion vectors
F
If mvy
F＜D
yT
y, show that the vertical direction overall situation relative motion vectors of present image is not obvious, the vertical component MVY (i)=0 of overall relative motion vectors is arranged; Otherwise show vertical component MVY (the i)=mvy that has overall relative motion vectors
F
For given threshold value T
xAnd T
yDecide according to concrete the application respectively, recommend 1≤T here
x, T
y≤ 2.
(4) the motion vector computation module of compensate for jitter
That this module is imported is overall relative motion vectors MVX (i) and MVY (i), the wherein 1≤i≤F-1 of preceding F-1 frame.Output is the image motion vector CMVX (F) and the CMVY (F) of the compensate for jitter of present frame.
A kind of method commonly used adopts curve fit, supposes that the data that preceding F-1 frame is arranged can determine a multinomial of P time
P(x)＝w
Px
P+w
P-1x
P-1+...+w
0
Here x is relative frame number, and first frame that we can select the F-1 frame is that 1, the second frame is that 2, the F-1 frames are F-1, and present frame is F.Output P (x) is the overall relative motion vectors of level or vertical direction.The p here sets according to actual conditions, for describing the exponent number of global motion vector track.For multiple normal global motion pattern, as at the uniform velocity taking photo by plane, become from speed 0 and at the uniform velocity to take photo by plane etc. and can describe by a multinomial of P time.As when the global motion pattern when at the uniform velocity taking photo by plane, p gets 1, when even the acceleration, p gets 2.Usually when calculating, the p value requires to choose according to the motion of actual support, and the motion of the high more description of p is also just complicated more, and general p≤5 just can be satisfied common motion and required.
By match, respectively the overall relative motion vectors of the present frame of horizontal direction and vertical direction is predicted that can obtain predicted current frame overall situation relative motion vectors is MVX (F) and MVY (F) to preceding F-1 two field picture.
The variance of F frame overall situation relative motion vectors is respectively before calculating simultaneously: DMVX (F) and DMVY (F), and, can determine the motion vector of compensate for jitter by following formula for COEFFICIENT K given in advance:
If | MVX (F)-MVX (F) |＞KDMVX (F), represent that then may there be shake in the present level direction, then needs to compensate, CMVX (F)=MVX (F)-MVX (F) is then arranged, otherwise think not have shake, belong to normal moving range, CMVX (F)=0 is then arranged.
If | MVY (F)-MVY (F) |＞KDMVY (F), represent that then may there be shake in current vertical direction, then needs to compensate, CMVY (F)=MVY (F)-MVY (F) is then arranged, otherwise think not have shake, belong to normal moving range, CMVY (F)=0 is then arranged.
The COEFFICIENT K here is recommended as 1≤K≤2.
As previously mentioned, because present embodiment only just carries out effective jitter compensation in the jitter compensation scope, therefore, the judgement formula
-XMAX≤CMVX(F)≤XMAX
-YMAX≤CMVY(F)≤YMAX
Whether set up,, show that the shake total size is positioned at maximum compensation range, directly exports the horizontal component CMVX (F) and the vertical component CMVY (F) of the motion vector of compensate for jitter if set up.If one of them is false or the two all is false, promptly in the horizontal direction and/or vertical direction exceeded maximum compensation range, then with described maximum compensating motion vector value XMAX and/or YMAX horizontal component and/or vertical component output as the motion vector of described compensate for jitter.
(5) jitter compensation module
The input of this module be the motion vector CMVX (F) of compensate for jitter and CMVY (F) and present frame big or small be the total pixel image of M * N, according to the effective pixel area of the M ' * N ' size of motion vector cutting relevant position, thereby be output as effective pixel area after the anti-shake compensation.Be specially:
At first, be central point (as the point of the o among Fig. 5) with the center of total pixel image of present frame, obtain compensation central point (as the o among Fig. 5 " point) according to the motion vector of compensate for jitter;
Then, be the center with described compensation central point (as the o among Fig. 5 " point), according to described valid pixel image size M ' * N ', on total pixel image of present frame, determine the clipping region;
At last, exporting image pixel in the described clipping region is effective image after the anti-shake compensation.
(6) overall relative motion vectors correcting module
Owing to carried out anti-shake compensation,, therefore need revise the overall relative motion vectors of F frame so the F frame with respect to the relative motion position of F-1 frame variation has taken place.
The overall relative motion vectors MVX (F) and the MXY (F) of the motion vector CMVX (F) of the compensate for jitter of this module input present frame and the present frame of CMVY (F) and estimation gained, thereby according to the following formula correction,
MVX(F)＝MVX(F)-CMVX(F)
MVY(F)＝MVY(F)-CMVY(F)，
And export described overall relative motion vectors memory module to and store
(7) overall relative motion vectors memory module
Owing to need the overall relative motion vectors of front F frame in the calculating of anti-shake compensating motion vector.Therefore the overall relative motion vectors of F frame before preserving by Global Vector Memory.
(8) reference frame storing module
Owing to adopted the method for estimation, therefore the image that needs former frame then is kept at the former frame image in the reference frame storing module as the reference frame, is used for the estimation of current frame image.
The anti-shake example of a reality as shown in Figure 7.The frame preface that Digital Video is taken is Frame 1, Frame2, and Frame 3, and Frame 4.Generation along with shake constantly compensates, but the region limits of compensation is in the scope of total pixel region.
The advantage of present embodiment is to need not to add motion sensor device based on the anti-fluttering method of estimation, and carries out the prediction of motion vector from the Digital Image Processing angle merely, realizes anti-shake.Present embodiment eliminated since video camera follow shot situation hypograph owing to the anti-shake jumping phenomena that causes.
It is anti-shake that the anti-shake technology of present embodiment is based on dynamic image, can be applied to equipment such as camera, Digital Video.
Claims (9)
1. the device of a video electronic flutter-proof comprises the motion vector computation module and the jitter compensation module of estimation computing module, overall relative motion vectors computing module, overall relative motion vectors memory module, compensate for jitter, wherein,
Described estimation computing module is imported the image of present frame and reference frame, and calculates the relative motion vectors of the relative reference frame of search block in the present frame according to motion estimation algorithm;
Described overall relative motion vectors computing module is imported the relative motion vectors from the search block of the present frame of described estimation computing module, thereby obtains the overall relative motion vectors of present frame according to the relative motion vectors of each search block; After described overall relative motion vectors computing module is imported the relative motion vectors of each search block, thereby the frequency of the relative motion vectors amplitude of described each search block of statistics obtains the relative motion vectors value that wherein frequency is the highest, and calculates the variance of described relative motion vectors; Judge then whether the highest relative motion vectors value of described frequency has surpassed the variance of the relative motion vectors of search block and the product of given threshold value in the image, if do not surpass, the overall relative motion vectors that then shows current frame image is not obvious, and the overall relative motion vectors of putting present frame is 0; Otherwise, with the overall relative motion vectors of the highest relative motion vectors value of described frequency as present frame;
Described overall relative motion vectors memory module is used to store the overall relative motion vectors of present frame some two field pictures before;
The motion vector computation module of described compensate for jitter is according to the overall relative motion vectors that obtains present frame from the overall relative motion vectors prediction of some frames before the present frame of storing in the overall relative motion vectors memory module, and overall relative motion vectors and the motion vector computation module of described compensate for jitter of calculating the present frame of overall relative motion vectors computing module output are predicted the difference of the overall relative motion vectors of the present frame that obtains, thereby judge whether to exist shake according to this difference, when having shake, the motion vector of shaking by way of compensation with described difference, when not having shake, the motion vector that makes compensate for jitter is 0; And whether the motion vector of judging described compensate for jitter is in the determined maximum compensation range of predefined maximum compensating motion vector value, if exceed this scope, then with the motion vector of described maximum compensating motion vector value as described compensate for jitter; The last motion vector of compensate for jitter of exporting again is to described jitter compensation module;
Total pixel image of described jitter compensation module input present frame, and according to the motion vector and the predefined valid pixel image size of described compensate for jitter, cutting obtains the effective image after the anti-shake compensation of present frame from total pixel image of present frame.
2. install according to claim 1, it is characterized in that: also comprise the pre-cut module, be used to import the overall relative motion vectors of each frame before total pixel image of present frame and the present frame, thereby obtain the absolute displacement value of the former frame image of present frame with respect to total pixel image center according to the overall relative motion vectors of each frame; And with this absolute displacement value definite center, clipping region on total pixel image of present frame, again according to the image size of predefined valid pixel, determine the clipping region with the center, clipping region, total pixel image to present frame carries out cutting at last, thereby the output size is the current frame video image of described valid pixel image size.
3. install as claimed in claim 1 or 2, it is characterized in that: also comprise overall relative motion vectors correcting module, be used to import the overall relative motion vectors of the present frame of the motion vector of compensate for jitter of present frame and estimation gained, thereby, and export described overall relative motion vectors memory module to and store with the motion vector of described overall relative motion vectors and compensate for jitter overall relative motion vectors as revised present frame.
4. as device as described in the claim 3, it is characterized in that: also comprise the reference frame storing module, be used for effective image of the present frame after the input jiffer compensation, reference frame when calculating as the next frame estimation.
5. as device as described in the claim 4, it is characterized in that: the motion vector computation module of described compensate for jitter adopts the overall relative motion vectors of the method prediction present frame of curve fit, is specially:
At first, according to the overall relative motion vectors of preceding some frames of present frame, determine a multinomial of p time:
P(x)＝w
px
p+w
p-1x
p-1+...+w
0
Wherein, x is relative frame number, and P (x) is that relative frame number is the overall relative motion vectors of the picture frame of x, and p is determined by the global motion pattern for describing the exponent number of overall relative motion vectors track;
Then, adopt described polynomial computation to obtain the overall relative motion vectors of present frame, be the overall relative motion vectors that forecasting institute gets.
6. as device as described in the claim 5, it is characterized in that: the motion vector computation module of described compensate for jitter judges whether to exist the method for shake to be specially according to the difference of the overall relative motion vectors of estimation and two kinds of method gained of prediction: at first, and the variance of the overall relative motion vectors of preceding some frames of calculating present frame; If: | the overall relative motion vectors of the present frame that the overall relative motion vectors-estimation of the present frame that prediction obtains obtains |＞K
*The variance of the overall relative motion vectors of preceding some frames of present frame represents that then there is shake in present frame, otherwise thinks and do not have shake, wherein 1≤K≤2.
7. as device as described in the claim 6, it is characterized in that: described jitter compensation module is on total pixel image of present frame, according to the motion vector and the predefined valid pixel image size of compensate for jitter, the step of the effective image after the anti-shake compensation of cutting acquisition present frame specifically comprises:
At first, be central point with the center of total pixel image of present frame, obtain the compensation central point according to the motion vector of compensate for jitter;
Then, be the center with described compensation central point, according to described valid pixel image size, on total pixel image of present frame, determine the clipping region;
At last, exporting image pixel in the described clipping region is effective image after the anti-shake compensation.
8. as device as described in the claim 7, it is characterized in that: the processing to overall relative motion vectors in described each module is all carried out from horizontal direction and vertical direction respectively, thereby the overall relative motion vectors of the final present frame that obtains is horizontal component and the vertical component by overall relative motion vectors to be constituted, the motion vector of the jitter compensation that is obtained also is made of its horizontal component and vertical component, and then to obtain effective image also be to carry out from horizontal direction and vertical direction respectively to the judgement of shake and cutting.
9. install according to claim 1, it is characterized in that: described search block adopts all macro blocks or the piece in the two field picture, perhaps adopt the data block of the pre-sizing in the two field picture, and described data block keeps at a certain distance away each other.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2005101080362A CN100389601C (en) | 2005-10-09 | 2005-10-09 | Video electronic flutter-proof device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2005101080362A CN100389601C (en) | 2005-10-09 | 2005-10-09 | Video electronic flutter-proof device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1744673A CN1744673A (en) | 2006-03-08 |
| CN100389601C true CN100389601C (en) | 2008-05-21 |
Family
ID=36139846
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2005101080362A Active CN100389601C (en) | 2005-10-09 | 2005-10-09 | Video electronic flutter-proof device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100389601C (en) |
Families Citing this family (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7952612B2 (en) * | 2006-06-22 | 2011-05-31 | Nokia Corporation | Method and system for image construction using multiple exposures |
| CN101246250B (en) * | 2008-01-07 | 2010-09-08 | 北京思比科微电子技术有限公司 | Anti-shake focusing method and system |
| CN101231456B (en) * | 2008-01-31 | 2010-11-24 | 北京航空航天大学 | Method for correcting panorama video jitter using angle sensor |
| EP2277306A4 (en) * | 2008-04-22 | 2014-07-16 | Core Logic Inc | Apparatus and method for correcting moving image wavering |
| JP2011128978A (en) * | 2009-12-18 | 2011-06-30 | Sony Corp | Information processor, information processing method and program |
| CN102340619A (en) * | 2010-07-21 | 2012-02-01 | 杭州华三通信技术有限公司 | Mobile detection method and device |
| CN102497494A (en) * | 2011-12-09 | 2012-06-13 | 首都师范大学 | FPGA (Field Programmable Gate Array)-based motion estimation module in high-speed image stabilizing system |
| CN103248855B (en) * | 2012-02-07 | 2016-12-14 | 北京同步科技有限公司 | Course recording system based on fixing seat in the plane, video process apparatus and record class method |
| CN103152522A (en) * | 2013-02-25 | 2013-06-12 | 东莞宇龙通信科技有限公司 | Method and terminal for adjusting shooting position |
| CN104618627B (en) * | 2014-12-31 | 2018-06-08 | 小米科技有限责任公司 | Method for processing video frequency and device |
| KR102449354B1 (en) * | 2016-02-04 | 2022-09-30 | 삼성전기주식회사 | Image stablization module and camera module |
| CN106331480B (en) * | 2016-08-22 | 2020-01-10 | 北京交通大学 | Video image stabilization method based on image splicing |
| CN106657758B (en) * | 2016-09-26 | 2020-11-27 | Oppo广东移动通信有限公司 | Photographing method and device and terminal equipment |
| CN110674665B (en) * | 2018-07-03 | 2023-06-30 | 杭州海康威视系统技术有限公司 | Image processing method and device, forest fire prevention system and electronic equipment |
| CN110771144B (en) | 2018-08-29 | 2021-11-16 | 深圳市大疆创新科技有限公司 | Control method and device of electronic equipment, electronic equipment and storage medium |
| CN110072049B (en) * | 2019-03-26 | 2021-11-09 | Oppo广东移动通信有限公司 | Image processing method and device, electronic equipment and computer readable storage medium |
| CN109922267A (en) * | 2019-04-01 | 2019-06-21 | 珠海全志科技股份有限公司 | Image stabilization processing method, computer installation and computer readable storage medium based on gyro data |
| CN110529896B (en) * | 2019-08-09 | 2021-09-07 | 杭州九阳小家电有限公司 | Correction method for smoke detection of range hood and range hood |
| CN112203091B (en) * | 2020-12-04 | 2021-05-04 | 浙江智慧视频安防创新中心有限公司 | Motion vector prediction method, system and computer medium based on quadratic polynomial |
| CN112367475B (en) * | 2021-01-15 | 2021-03-30 | 上海闪马智能科技有限公司 | Traffic incident detection method and system and electronic equipment |
| CN113163120A (en) * | 2021-04-21 | 2021-07-23 | 安徽清新互联信息科技有限公司 | Transformer-based video anti-shake method |
| CN113411500B (en) * | 2021-06-18 | 2024-01-12 | 上海盈方微电子有限公司 | Global motion vector estimation method and electronic anti-shake method |
| CN115242967A (en) * | 2022-06-07 | 2022-10-25 | 浙江华感科技有限公司 | Imaging anti-shake method, imaging anti-shake apparatus, image pickup device, and readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5909242A (en) * | 1993-06-29 | 1999-06-01 | Sanyo Electric Co., Ltd. | Video camera with electronic picture stabilizer |
| CN1330493A (en) * | 2000-06-28 | 2002-01-09 | 三星电子株式会社 | Decoder with digital image stability function and image stability method |
-
2005
- 2005-10-09 CN CNB2005101080362A patent/CN100389601C/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5909242A (en) * | 1993-06-29 | 1999-06-01 | Sanyo Electric Co., Ltd. | Video camera with electronic picture stabilizer |
| CN1330493A (en) * | 2000-06-28 | 2002-01-09 | 三星电子株式会社 | Decoder with digital image stability function and image stability method |
Non-Patent Citations (8)
| Title |
|---|
| "机载电子稳像技术研究". 钟平.博士论文，中国科学院长春光学与精密机械与物理研究所. 2004 |
| "机载电子稳像技术研究". 钟平.博士论文，中国科学院长春光学与精密机械与物理研究所. 2004 * |
| 基于全搜索块匹配法的电子图像稳定. 冯驰，张怡.计算机应用，第23卷第6期. 2004 |
| 基于全搜索块匹配法的电子图像稳定. 冯驰，张怡.计算机应用，第23卷第6期. 2004 * |
| 实现图像稳定的快速算法. 葛卫龙，雷选华.红外与激光工程，第33卷第1期. 2004 |
| 实现图像稳定的快速算法. 葛卫龙，雷选华.红外与激光工程，第33卷第1期. 2004 * |
| 航空光电侦察平台关键技术及其发展. 贾平，张葆.光学精密工程，第11卷第1期. 2003 |
| 航空光电侦察平台关键技术及其发展. 贾平，张葆.光学精密工程，第11卷第1期. 2003 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1744673A (en) | 2006-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100389601C (en) | Video electronic flutter-proof device | |
| CN100421455C (en) | Video electronic flutter-proof method | |
| CN110692083B (en) | Block-matched optical flow and stereoscopic vision for dynamic vision sensor | |
| US9124807B2 (en) | Imaging apparatus, control method therefor, and storage medium | |
| KR101159082B1 (en) | Robust camera pan vector estimation using iterative center of mass | |
| CN101690171B (en) | Imaging device, integrated circuit, and imaging method | |
| US20100271554A1 (en) | Method And Apparatus For Motion Estimation In Video Image Data | |
| CN102870403A (en) | Image processing device, image capture device, program, and image processing method | |
| CN100574380C (en) | Formation method and imaging device | |
| CN102067607B (en) | Image synthesizing and encoding method, image synthesizing and encoding device, and imaging system | |
| CN100551057C (en) | A kind of method and encoder of realizing video code anti-shake based on dynamic image | |
| CN102450019A (en) | Image processing device, image generating system, method, and program | |
| CN109328454A (en) | Image processing apparatus | |
| CN103634513A (en) | Image processing apparatus and control method thereof | |
| CN1965583A (en) | Method of searching for a global motion vector | |
| CN110351508B (en) | Anti-shake processing method and device based on video recording mode and electronic equipment | |
| US5296925A (en) | Movement vector detection device | |
| CN113014817B (en) | Method and device for acquiring high-definition high-frame video and electronic equipment | |
| JP6508926B2 (en) | Image pickup apparatus and control method of image pickup apparatus | |
| US20090322956A1 (en) | System and method for motion estimation of digital video using multiple recursion rules | |
| CN105915835B (en) | One kind can hard-wired frame frequency lifting method | |
| CN101189870A (en) | Motion stabilization | |
| JP2011259390A (en) | Flicker detector | |
| KR100748174B1 (en) | Apparatus for detecting and correcting motion by hand in moving pictures | |
| RU2337501C2 (en) | Method for blur compensation of moving object image and device for implementation of method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20171219 Address after: 519031 Guangdong Zhuhai Hengqin Baohua Road No. 6, room 105, -23898 (central office) Patentee after: Zhongxing Technology Co., Ltd. Address before: 100083, Haidian District, Xueyuan Road, Beijing No. 35, Nanjing Ning building, 15 Floor Patentee before: Beijing Vimicro Corporation |
|
| CP01 | Change in the name or title of a patent holder | ||
| CP01 | Change in the name or title of a patent holder |
Address after: 519031 -23898, 105 room 6, Baohua Road, Hengqin New District, Zhuhai, Guangdong (centralized office area) Patentee after: Mid Star Technology Limited by Share Ltd Address before: 519031 -23898, 105 room 6, Baohua Road, Hengqin New District, Zhuhai, Guangdong (centralized office area) Patentee before: Zhongxing Technology Co., Ltd. |