CN102098440B - Electronic image stabilizing method and electronic image stabilizing system aiming at moving object detection under camera shake - Google Patents

Abstract

The invention discloses an electronic image stabilizing method and an electronic image stabilizing system aiming at moving object detection under camera shake. The method comprises the following steps: carrying out guassian blur on the original image, lowering the resolution ratio and determining three layers of images; determining a first translational motion vector corresponding to an image with the lowest resolution ratio positioned at the highest layer based on a blocking gray projection method; compensating an interlayer image based on the acquired first translational motion vector; determining a second translational motion vector corresponding to the image positioned in the interlayer based on a block match algorithm; compensating the original image with the highest resolution ratio positioned at the lowest layer based on the first translational motion vector and the second translational motion vector; and smoothing and compensating the motion by utilizing a feature matching method. The method and system provided by the invention have good robustness to the dynamic background, the noise, and the consistency of motion, rotation and background of large-sized images, have superiority on the image quality, the visual effect and the calculation speed, and satisfy the requirements of the moving object detection under the camera shake after the image is stabilized.

Description

Electronic image stabilization method and system for moving object detection under the DE Camera Shake

Technical field

The present invention relates to the moving object detection under the dynamic scene image sequence, relate in particular to a kind of electronic image stabilization method for moving object detection under the DE Camera Shake and system.

Background technology

Along with the progress of human society and the development of science and technology, utilize computer to realize that human visual performance becomes one of problem the most popular in the present computer realm.The research of Detection for Moving Target is an important subject of computer vision field.Moving object detection is exactly that the technology such as image sequence utilization Digital Signal Processing that comprise movable information are suitably processed, detect and extract the prospect that has relative motion in the image sequence with background, determine the position of moving target, it has had in many fields such as video monitoring, virtual reality, robot navigation, military aiming, tv edit, medical image analysis widely uses, and therefore has important use value and vast potential for future development.

But because the occasion of various Video Applications is not quite similar, the residing environment of moving target and background are ever-changing, and this adaptability and robustness to the moving object detection algorithm is had higher requirement.Wherein, the moving object detection under the DE Camera Shake is difficult point and the focus of field of video image processing research, also day by day becomes a large obstacle of video image processing system practicality and reliability.DE Camera Shake refers to because skew or the rotation that the camera lens that the factors such as wind, exterior vibration cause occurs.DE Camera Shake can cause generation skew or rotation between adjacent image, causes one by one correspondence of image pixel coordinate.

Summary of the invention

The object of the invention is to the needs of problems for the lower moving object detection of camera shake, take into full account moving target in the scene to the impact of the moving algorithm of video camera debounce, a kind of electronic image stabilization method for moving object detection under the DE Camera Shake and system are provided, based on the present invention, the consistency of dynamic background, noise, large scale image motion, rotation and the background that occurs in can the video to detection process of moving target has good robust property.

A kind of electronic image stabilization method for moving object detection under the DE Camera Shake of the present invention, comprise the steps: stratification step, original image to reference frame image and current frame image carries out respectively Gaussian Blur and reduces resolution, be respectively described reference frame image and current frame image and determine three tomographic images, comprising: be positioned at the highest original image of lowermost layer and resolution, intermediate layer image and be positioned at image top and that resolution is minimum; The first motion-estimation step, based on piecemeal Gray Projection method, determine described each piece zone that is arranged in top and the current frame image that resolution is minimum with respect to being positioned at each corresponding motion vector in piece zone top and reference frame image that resolution is minimum, and then obtain a plurality of motion vectors; In described a plurality of motion vectors, the regional corresponding motion vector of moving target may appear in removal, according to remaining motion vector computation the first translational motion vector; The second estimation and compensation process, based on described the first translational motion vector that obtains, the intermediate layer image of compensation current frame image; In the intermediate layer image of the described current frame image after compensation, remove the described zone that moving target may occur, the present frame intermediate layer image after obtaining to compensate; Utilize BMA, obtain present frame intermediate layer image after the described compensation with respect to the second motion translation vector of reference frame intermediate layer image; The 3rd estimation and compensation process based on the first translational motion vector and the second motion translation vector, compensate the described original image that is positioned at the highest current frame image of lowermost layer and resolution, obtain the present frame original image after the compensation; Utilize the characteristic matching method that the original image of reference frame image and the present frame original image after the described compensation are carried out motion smoothing and compensation.

Above-mentioned electronic image stabilization method, also be provided with behind preferred described the 3rd estimation and the compensation process: the moving target extraction step, based on the original image motion smoothing of described current frame image and the result after the compensation, with respect to the original image detection moving target of reference frame image.

Above-mentioned electronic image stabilization method, in the preferred described stratification step, described intermediate layer image and be positioned at top and image that resolution is minimum and obtain based on the mode of mean filter.

Above-mentioned electronic image stabilization method in preferred described the second estimation and the compensation process, utilizes 2 times of described the first translational motion vector that described intermediate layer image is compensated.

Above-mentioned electronic image stabilization method, in preferred described the 3rd estimation and the compensation process, based on affine model, carry out motion smoothing and compensation, wherein, the parameter of affine model is determined in the following way: original image and the present frame original image after the described compensation to reference frame image carry out yardstick invariant features converting characteristic coupling, find out corresponding points, obtain kinematic parameter in the substitution affine model, realize the original image translation of current frame image and the estimation that rotatablely moves; The parameter that the based on motion estimation draws is carried out level and smooth and compensation to the present frame original image sequence after the described compensation; Wherein, affine model is determined by following formula:

$\left(\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\end{array}\right)=\left(\begin{array}{cc}{a}_1& a_2\\ a_3& a_4\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)+\left(\begin{array}{c}{d}_1\\ {\mathrm{<figure-callout\; id="2"\; label="d"\; filenames="HDA0000038700550000031.png,HDA0000038700550000042.png"\; state="\{\{state\}\}">d</figure-callout>}}_2\end{array}\right),$

Wherein A certain pixel is at the coordinate of reference frame and present frame, d in the difference presentation video ₁, d ₂Expression

With respect to

Side-play amount at x axle and y axle.a ₁～a ₄Expression rotation parameter and convergent-divergent among a small circle.

On the other hand, the invention also discloses a kind of electronic steady image system for moving object detection under the DE Camera Shake, comprising: hierarchical block, the first motion estimation module, the second estimation and compensating module and the 3rd estimation and compensating module.Wherein:

Hierarchical block is used for the original image of reference frame image and current frame image is carried out respectively Gaussian Blur and reduces resolution, be respectively described reference frame image and current frame image and determine three tomographic images, comprising: be positioned at the highest original image of lowermost layer and resolution, intermediate layer image and be positioned at image top and that resolution is minimum; The first motion estimation module is used for based on piecemeal Gray Projection method, determine described be arranged in top and current frame image that resolution is minimum with respect to being positioned at each corresponding motion vector in piece zone of reference frame image top and that resolution is minimum, and then obtain a plurality of motion vectors; In described a plurality of motion vectors, the regional corresponding motion vector of moving target may appear in removal, according to remaining motion vector computation the first translational motion vector; The second estimation and compensating module are used for based on described the first translational motion vector that obtains, the intermediate layer image of compensation current frame image; In the intermediate layer image of the described current frame image after compensation, remove the described zone that moving target may occur, the present frame intermediate layer image after obtaining to compensate; Utilize BMA, obtain present frame intermediate layer image after the described compensation with respect to the second motion translation vector of reference frame intermediate layer image; The 3rd estimation and compensating module be based on the first translational motion vector and the second motion translation vector, compensates the described original image that is positioned at the highest current frame image of lowermost layer and resolution, obtains the present frame original image after the compensation; Utilize the characteristic matching method that the original image of reference frame image and the present frame original image after the described compensation are carried out motion smoothing and compensation.

Above-mentioned electronic steady image system, also be connected with behind preferred described the 3rd estimation and the compensating module: the moving target extraction module, be used for based on the original image motion smoothing of described current frame image and the result after the compensation, with respect to the original image detection moving target of reference frame image.

Above-mentioned electronic steady image system, in the preferred described hierarchical block, described intermediate layer image and be positioned at top and image that resolution is minimum and obtain based on the mode of mean filter.

Above-mentioned electronic steady image system in preferred described the second estimation and the compensating module, utilizes 2 times of described the first translational motion vector that described intermediate layer image is compensated.

Above-mentioned electronic steady image system, in preferred described the 3rd estimation and the compensating module, based on affine model, carry out motion smoothing and compensation, wherein, the parameter of affine model is determined in the following way: original image and the present frame original image after the described compensation to reference frame image carry out yardstick invariant features converting characteristic coupling, find out corresponding points, obtain kinematic parameter in the substitution affine model, realize the original image translation of current frame image and the estimation that rotatablely moves; The parameter that the based on motion estimation draws is carried out level and smooth and compensation to the present frame original image sequence after the described compensation; Wherein, affine model is determined by following formula:

$\left(\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\end{array}\right)=\left(\begin{array}{cc}{a}_1& a_2\\ a_3& a_4\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)+\left(\begin{array}{c}{d}_1\\ {\mathrm{<figure-callout\; id="2"\; label="d"\; filenames="HDA0000038700550000031.png,HDA0000038700550000042.png"\; state="\{\{state\}\}">d</figure-callout>}}_2\end{array}\right),$

Wherein

A certain pixel is at the coordinate of reference frame and present frame, d in the difference presentation video ₁, d ₂Expression With respect to

Side-play amount at x axle and y axle.a ₁～a ₄Expression rotation parameter and convergent-divergent among a small circle.

In terms of existing technologies, the present invention is by adopting the multiresolution layered approach, minutes three layers carry out different estimation and compensation method to image, and solved a lot of interference problem in the steady picture process: characteristic matching method and piecemeal Gray Projection method are relatively more responsive to change of background such as dynamic background, noises; The block-based histogram feature of BMA adapts to the problems such as dynamic background, noise, but piece coupling computation complexity is high, affected by the background area consistency; Piecemeal Gray Projection method estimated motion vector has been processed the large scale image motion, and has avoided the impact of moving target, but responsive to change of background, and image reduces resolution and suppressed local faint change of background.Piecemeal Gray Projection method and BMA all can not be processed the rotation of image, utilize the rotating vector of characteristic matching method estimated image.

The present invention can both have good robust property for the consistency of the dynamic background that occurs in the video, noise, large scale image motion, rotation and background.And with respect to other algorithms, certain superiority is arranged on picture quality, visual effect, computational speed, can satisfy the actual demand of the lower moving object detection of camera shake behind the steady picture.

Description of drawings

Fig. 1 is the flow chart of steps that the present invention is directed to the electronic image stabilization method embodiment of moving object detection under the DE Camera Shake;

Fig. 2 is the multiresolution hierarchical diagram schematic diagram of image;

Fig. 3 is that layering multiresolution overall motion estimation vector is estimated the flow graph schematic diagram;

Fig. 4 is that the low-resolution image piecemeal utilizes sciagraphy to obtain the motion vector schematic diagram;

Fig. 5 carries out as a result figure of electronic steady image foreground detection afterwards;

Fig. 6 compares schematic diagram for rectification and the target detection that migrated image occurs;

Fig. 7 is that the Y-PSNR of sequence image compares schematic diagram;

Fig. 8 is the structured flowchart that the present invention is directed to the electronic steady image system embodiment of moving object detection under the DE Camera Shake;

Fig. 9 is the structured flowchart that the present invention is directed to the electronic steady image system embodiment of moving object detection under the DE Camera Shake.

Embodiment

For above-mentioned purpose of the present invention, feature and advantage can be become apparent more, the present invention is further detailed explanation below in conjunction with the drawings and specific embodiments.

With reference to Fig. 1, Fig. 1 is the flow chart of steps that the present invention is directed to the electronic image stabilization method embodiment of moving object detection under the DE Camera Shake, comprise the steps: stratification step S110, original image to reference frame image and current frame image carries out respectively Gaussian Blur and reduces resolution, be respectively three tomographic images of determining of reference frame image and current frame image, comprise: be positioned at the highest original image of lowermost layer and resolution, intermediate layer image and be positioned at image top and that resolution is minimum; The first motion-estimation step S120, based on piecemeal Gray Projection method, determine to be arranged in top and current frame image that resolution is minimum with respect to each corresponding motion vector in piece zone of reference frame image, and then obtain a plurality of motion vectors; In a plurality of motion vectors, the regional corresponding motion vector of moving target may appear in removal, according to remaining motion vector computation the first translational motion vector; The second estimation and compensation process S130, based on the first translational motion vector that obtains, the intermediate layer image of compensation current frame image; In the intermediate layer image of the current frame image after compensation, the zone of moving target may appear in removal, for remaining zone, utilize BMA to be compensated and remove the intermediate layer current frame image that motion target area may occur with respect to the second motion translation vector of reference frame image; The 3rd estimation and compensation process S140 based on the first translational motion vector and the second motion translation vector, compensate the described original image that is positioned at the highest current frame image of lowermost layer and resolution, obtain the present frame original image after the compensation; Utilize the characteristic matching method that the original image of reference frame image and the present frame original image after the described compensation are carried out motion smoothing and compensation.

The present embodiment all has good robust property for the consistency of the dynamic background that occurs in the video, noise, large scale image motion, rotation and background; And with respect to other algorithms, certain superiority is arranged on picture quality, visual effect, computational speed, can satisfy the actual demand of the lower moving object detection of camera shake behind the steady picture.

The invention will be further described below in conjunction with the drawings and specific embodiments.

The present invention carries out image Gaussian Blur and reduces resolution, is divided into three layers, and three layers of image in different resolution are carried out respectively different processing.Adopt respectively Gray Projection method, BMA, improving one's methods of characteristic matching method that each tomographic image is carried out estimation and compensation.The below simply introduces respectively this three kinds of methods.

1, Gray Projection method

The Gray Projection method is to utilize the ranks intensity profile to carry out the motion estimation algorithm of related operation.This algorithm utilizes the ranks Gray Projection curve of image to do the motion vector that related operation obtains image.Algorithm mainly is comprised of image mapped and related operation.

1) image mapped

After using histogram equalization to carry out preliminary treatment to the two dimensional image of each frame input, be mapped to two independently one dimension waveforms.Projecting method can be described by formula (3) (4),

$G_K\left(i\right)=\underset{j}{\mathrm{\&Sigma;}}{G}_k(i,j)---\left(3\right)$

$G_K\left(j\right)=\underset{i}{\mathrm{\&Sigma;}}{G}_k(i,j)---\left(4\right)$

G in the formula _K(i), G _K(j) be respectively that k two field picture i is capable, the gray value sum of each pixel of j row, G _K(i, j) is the grey scale pixel value of (i, j) position on the k two field picture.

2) correlation computations

The projection waveform of k two field picture projection waveform and reference picture is done cross-correlation calculation.Wherein the row correlation computations can be described by formula (5),

$C\left(w\right)={\mathrm{\&Sigma;}}_{j=1}^N{[{\mathrm{Col}}_k(j+w)-{\mathrm{Col}}_r(M+j)]}^2---\left(5\right)$

In the formula, 1≤w≤2M+1, Col _k(j), Col _r(j) be respectively the Gray Projection value of the j row of k two field picture and reference frame image, N is the length of row, and M is that displacement vector is with respect to the search width of reference frame in a side.

2, block matching algorithm

Based on the estimation of piece coupling, refer to picture frame is divided into the identical a series of sub-blocks of size, each sub-block size is N * N.For a sub-block in the reference frame, in the k frame, seek the most similar sub-block identical with its size, the matching criterior commonly used of seeking match block has three kinds, i.e. minimum average B configuration absolute difference (MAD), least mean-square error (MSE) and Normalized Cross Correlation Function (NCCF).This paper adopts least mean-square error (MSE) criterion to mate, can be described by formula (6),

$\mathrm{MSE}\left(d\right)=\frac{1}{N_x\&times;N_y}\underset{x=1}{\overset{N_x}{\mathrm{\&Sigma;}}}\underset{y=1}{\overset{N_y}{\mathrm{\&Sigma;}}}{[I_k(x+d_x,y+d_y)-I_t(x,y)]}^2---\left(6\right)$

Wherein, I _k(x, y), I _t(x, y) represents k, the gray value that t two field picture mid point (x, y) is located, d=(d _x, d _y) be translation parameters.

3, characteristic matching method

Based on the motion estimation algorithm of characteristic matching selected characteristic in reference frame at first, then in subsequent frame, feature is positioned, utilize at last the feature set corresponding relation, ask for globe motion parameter.The algorithm basic step is described below:

A) feature detection: a certain two field picture in the image sequence is processed, extracted characteristics of image, number of features is m.

B) characteristic matching: the spatial domain relation between use characteristic identifier, similarity measurement and feature, determine that feature is in the positional information of current frame image.

C) transformation model is estimated: the correspondence position of characteristic quantity concerns simultaneous equations in the substitution motion model, separates the overdetermined equation group, obtains global motion vector.

Below in conjunction with a concrete example, the present invention realizes that the method for moving object detection is described in detail to foundation:

Step 1: stratification step

The impact of moving in order to reduce image local motion and moving target, the thought of image pyramid layering is introduced in invention, divides three layers to reduce resolution to original image.Because low resolution image can the smoothed image local motion, the noise reduction impact improves the algorithm real-time, and in low-resolution image, the low-angle image rotation that DE Camera Shake causes can be left in the basket, and the image overall motion can be reduced to translational motion.As shown in Figure 2, lowermost layer is original image, reduces gradually resolution from bottom to high level, and the simple mean filter of normal operation can be described by formula (7):

$f_k^{(l+1)}(i,j)=\frac{1}{4}\underset{m=0}{\overset{1}{\mathrm{\&Sigma;}}}\underset{n=0}{\overset{1}{\mathrm{\&Sigma;}}}{f}_k^{\left(l\right)}(2i+m,2j+n)1=\mathrm{0,1},...N---\left(7\right)$

In the formula (7),

Represent that (1+1) layer of k frame is at the pixel value of (i, j) position.

K two field picture for input.

Step 2: the first motion-estimation step and the second estimation and compensation process

This step is low-resolution image estimation and compensation.Take the image three-decker as the basis, Fig. 3 has provided the flow chart based on the image overall estimation of three-decker.Invention will be to doing concrete elaboration in the top algorithm block diagram, below be method in the specific implementation of three tomographic images:

1. the 3rd layer of low-resolution image carried out piecemeal, then each piecemeal is utilized sciagraphy, estimate the motion vector of each sub-block.As shown in Figure 4, A, B figure is respectively the image of reference frame and present frame among the figure, certain skew has occured with respect to reference frame image in present frame, C figure has represented to utilize the motion vector of the subregion that piecemeal Gray Projection method obtains, wherein because the existence of dolly has caused the motion vector of dolly place subregion estimation and the vector generator relatively large deviation that other subregions are estimated.Each sub regions motion vector is added with average, the global motion vector MV (3) of formation, little and larger with the consistent motion vector difference vector of probability of occurrence has been eliminated in adding and process of vector.

2. the 2nd layer current frame image is carried out the global motion compensation of 2 * MV (3), and image is carried out piecemeal, partitioned mode is similar to the 3rd layer, and the number of piece is identical, but size is twice, but removes the impact that the edge causes in the reality, and size is slightly smaller.Next find out the subregion corresponding with the 3rd straton piece at the 2nd layer, these several sub regions are carried out piece coupling, ask for motion vector and add and on average obtain motion vector MV (2).Utilize the motion estimation vectors MV (2) of the 2nd tomographic image that vector MV (3) is had the compensating action of refining.

Step 3: the 3rd estimation and compensation process

This step is original image estimation and compensation.At first utilize vector 4 * MV (3) and 2 * MV (2) to compensate to the 1st layer, because MV (3) and MV (2) vector have just reacted the translational motion of image, algorithm will add the estimation to the image rotation parameter in 1 layer estimation, algorithm utilizes yardstick invariant features conversion (SIFT) characteristic matching to find out corresponding points, obtain kinematic parameter in the substitution affine model, realize image translation and the estimation that rotatablely moves.We choose n to angular coordinate, in coordinate substitution affine model, can get the linear system equation:

N then can obtain (2n * 6) the rank system of linear equations on the real number field to matching characteristic, and solution of equations is the globe motion parameter of interframe.And the equation number 2n that obtains generally speaking＞6, i.e. overdetermined equation group.This equation group does not have the solution under the ordinary meaning, for this reason the demand equation group least square solution.Definition energy function ρ _x, ρ _y:

${\mathrm{\&rho;}}_x=\underset{i=0}{\overset{N}{\mathrm{\&Sigma;}}}{(a_1{x}_i+a_2{y}_i+d_1-x^{\&prime;})}^2---\left(9\right)$

${\mathrm{\&rho;}}_y=\underset{i=0}{\overset{N}{\mathrm{\&Sigma;}}}{(a_3{x}_i+a_4{y}_i+d_2-y^{\&prime;})}^2---\left(10\right)$

Then have:

$\frac{\&PartialD;{\mathrm{\&rho;}}_x}{\&PartialD;a_1}=\frac{{\&PartialD;\mathrm{\&rho;}}_x}{\&PartialD;a_2}=\frac{\&PartialD;{\mathrm{\&rho;}}_x}{\&PartialD;{\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="HDA0000038700550000042.png"\; state="\{\{state\}\}">d</figure-callout>}}_1}=\frac{\&PartialD;{\mathrm{\&rho;}}_y}{\&PartialD;a_3}=\frac{\&PartialD;{\mathrm{\&rho;}}_y}{\&PartialD;a_4}=\frac{\&PartialD;{\mathrm{\&rho;}}_y}{\&PartialD;{\mathrm{<figure-callout\; id="2"\; label="d"\; filenames="HDA0000038700550000031.png,HDA0000038700550000042.png"\; state="\{\{state\}\}">d</figure-callout>}}_2}=0---\left(11\right)$

Thereby can try to achieve parameter a ₁～a ₄, d ₁～d ₂, draw parameter a by estimation ₁～a ₄, d ₁～d ₂After, the substitution affine model can compensate image sequence, can be described by formula (12) (13).

$x=\frac{{a_4}^{*}{x}^{,}-{a_2}^{*}{y}^{,}-{{\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="HDA0000038700550000042.png"\; state="\{\{state\}\}">d</figure-callout>}}_1}^{*}{a}_4+{a_2}^{*}{d}_2}{{a_1}^{*}{a}_4-{a_2}^{*}{a}_3}---\left(12\right)$

$y=\frac{{a_3}^{*}{x}^{,}-{a_1}^{*}{y}^{,}-{{\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="HDA0000038700550000042.png"\; state="\{\{state\}\}">d</figure-callout>}}_1}^{*}{a}_3+{a_1}^{*}{d}_2}{{a_2}^{*}{a}_3-{a_1}^{*}{a}_4}---\left(13\right)$

In the formula

A certain pixel is at the coordinate of reference frame and present frame, d in the difference presentation video ₁, d ₂Expression

With respect to

Side-play amount at x axle and y axle.a ₁～a ₄Expression rotation parameter and convergent-divergent among a small circle.

Step 4: moving target extracts

Invention is adopted and is carried out the extraction of moving target based on the method for background subtraction, verifies steady picture effect.At first current frame image and background image are carried out calculus of differences, then difference image is carried out the binaryzation operation, if pixel value is greater than a certain threshold value in the difference image, judge that then this pixel belongs to motion target area, otherwise, just judge that this pixel belongs to the background area, target detection result as shown in Figure 5.

It is carrier that the present invention adopts 320 * 240 image sequence, and a large amount of experiments show, our method is for the moving object detection under the DE Camera Shake of indoor and outdoor, can reach surely to look like preferably the result.As shown in Figure 6, considered the impact of moving target on the moving algorithm of video camera debounce, the zone that may have moving target is removed, the result of detection is better than the result that the three kinds of algorithms in front are corrected, and wherein also contains the rotation of image among the 3rd width of cloth figure.Among Fig. 6, A figure expression reference frame image, B figure expression current frame image, C figure is that the Gray Projection method is corrected rear image and A desires to make money or profit with the result of calculus of finite differences detection, D figure is that BMA is corrected rear image and A desires to make money or profit with the result of calculus of finite differences detection, E figure is that yardstick invariant features converting characteristic matching method is corrected rear image and A desires to make money or profit with the result of calculus of finite differences detection, can find out that the three kinds of method testing results in front detect wrong more, F figure is that our algorithm is corrected rear image and A desires to make money or profit with the result of calculus of finite differences detection, detects wrong less in the testing result.Can be found out by the result, only use first three methods not consider moving target to the impact of algorithm, cause algorithm to estimate kinematic parameter generation deviation.

Fig. 7 has provided the Y-PSNR value of 5 two field pictures in the sequence image.Y-PSNR is a kind of assessment of performance standard, and is identical with mean square deviation in essence, its reflection be Y-PSNR between reference frame image and the current frame image, the Y-PSNR value is higher, the video stabilization effect is better.Y-PSNR is defined as follows:

$\mathrm{PNSR}=10\log \frac{{255}^2}{\mathrm{MSE}(I_k,I_t)}---\left(14\right)$

MSE (I wherein _k, I _t) mean square deviation, be defined as follows:

$\mathrm{MSE}(I_k,I_t)=\frac{1}{M\&times;N}\underset{x=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{Y=1}{\overset{N}{\mathrm{\&Sigma;}}}{[I_k(x,y)-I_t(x,y)]}^2---\left(15\right)$

Can find out from above result, the method that the present invention proposes has been considered the existence of moving target, can effectively process the video of the DE Camera Shake that contains moving target, and can high-qualityly carry out electronic steady image.Consistency for the dynamic background that occurs in the video, noise, large scale image motion, rotation and background can both have good robust property.With respect to other algorithms, certain superiority is arranged on picture quality, visual effect, computational speed, this scheme computing is simple and convenient, and reliability is high, and real-time is good, can satisfy the actual demand of the lower moving object detection of camera shake.

Other method the invention also discloses a kind of electronic steady image system for moving object detection under the DE Camera Shake.With reference to Fig. 8, Fig. 8 is the structured flowchart that the present invention is directed to the electronic steady image system embodiment of moving object detection under the DE Camera Shake, comprising: stratification step 80, the first motion-estimation step 82, the second estimation and compensation process 84 and the 3rd estimation and compensation process 86.

Wherein: hierarchical block 80 is used for the original image of reference frame image and current frame image is carried out respectively Gaussian Blur and reduces resolution, be respectively three tomographic images of determining of described reference frame image and current frame image, comprise: be positioned at the highest original image of lowermost layer and resolution, intermediate layer image and be positioned at image top and that resolution is minimum; The first motion estimation module 82 is used for based on piecemeal Gray Projection method, determine described be arranged in top and current frame image that resolution is minimum with respect to being positioned at each corresponding motion vector in piece zone of reference frame image top and that resolution is minimum, and then obtain a plurality of motion vectors; In described a plurality of motion vectors, the regional corresponding motion vector of moving target may appear in removal, according to remaining motion vector computation the first translational motion vector; The second estimation and compensating module 84 are used for based on the first translational motion vector that obtains, the intermediate layer image of compensation current frame image; In the intermediate layer image of the current frame image after compensation, the zone of moving target may appear in removal, obtains the first present frame intermediate layer image; Zone for remaining utilizes BMA, obtains the first present frame intermediate layer image with respect to the second motion translation vector of reference frame intermediate layer image; The 3rd estimation and compensating module 86 be based on the first translational motion vector and the second motion translation vector, compensates the described original image that is positioned at the highest current frame image of lowermost layer and resolution, obtains the present frame original image after the compensation; Utilize the characteristic matching method that the original image of reference frame image and the present frame original image after the described compensation are carried out motion smoothing and compensation.

With reference to Fig. 9, Fig. 9 is the structured flowchart for another embodiment of electronic steady image system of moving object detection under the DE Camera Shake.This embodiment also is connected with moving target extraction module 88 on the basis of Fig. 8, this module is used for detecting moving target based on the result after original image motion smoothing and the compensation.

In one embodiment, in the second estimation and the compensating module 84, utilize 2 times of the first translational motion vector that described intermediate layer image is compensated.

In one embodiment, in the 3rd estimation and the compensating module 86, also comprise and obtain the affine model parameter acquiring unit, this unit is used for:

Present frame original image after described reference frame original image and the described compensation is carried out yardstick invariant features converting characteristic coupling, find out corresponding points, obtain kinematic parameter in the substitution affine model, realize current frame image translation and the estimation that rotatablely moves; The parameter that the based on motion estimation draws is carried out level and smooth and compensation to the original image sequence; Wherein, affine model is determined by following formula:

$\left(\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\end{array}\right)=\left(\begin{array}{cc}{a}_1& a_2\\ a_3& a_4\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)+\left(\begin{array}{c}{d}_1\\ {\mathrm{<figure-callout\; id="2"\; label="d"\; filenames="HDA0000038700550000031.png,HDA0000038700550000042.png"\; state="\{\{state\}\}">d</figure-callout>}}_2\end{array}\right),$

Wherein A certain pixel is at the coordinate of reference frame and present frame, d in the difference presentation video ₁, d ₂Expression

With respect to

Side-play amount at x axle and y axle.a ₁～a ₄Expression rotation parameter and convergent-divergent among a small circle.

Above a kind of electronic image stabilization method and system for moving object detection under the DE Camera Shake provided by the present invention described in detail, used specific embodiment herein principle of the present invention and execution mode are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, all will change in specific embodiments and applications.In sum, this description should not be construed as limitation of the present invention.

Claims (8)

1. the electronic image stabilization method for moving object detection under the DE Camera Shake is characterized in that, comprises the steps:

Stratification step, original image to reference frame image and current frame image carries out respectively Gaussian Blur and reduces resolution, be respectively described reference frame image and current frame image and determine three tomographic images, comprising: be positioned at the highest original image of lowermost layer and resolution, intermediate layer image and be positioned at image top and that resolution is minimum;

The first motion-estimation step, based on piecemeal Gray Projection method, determine described each piece zone that is arranged in top and the current frame image that resolution is minimum with respect to being positioned at each corresponding motion vector in piece zone top and reference frame image that resolution is minimum, and then obtain a plurality of motion vectors; In described a plurality of motion vectors, the regional corresponding motion vector of moving target may appear in removal, according to remaining motion vector computation the first translational motion vector;

The second estimation and compensation process, based on described the first translational motion vector that obtains, the intermediate layer image of compensation current frame image; In the intermediate layer image of the described current frame image after compensation, remove the described zone that moving target may occur, the present frame intermediate layer image after obtaining to compensate; Utilize BMA, obtain present frame intermediate layer image after the described compensation with respect to the second motion translation vector of reference frame intermediate layer image;

The 3rd estimation and compensation process based on the first translational motion vector and the second motion translation vector, compensate the described original image that is positioned at the highest current frame image of lowermost layer and resolution, obtain the present frame original image after the compensation; Utilize the characteristic matching method that the original image of reference frame image and the present frame original image after the described compensation are carried out motion smoothing and compensation.

2. electronic image stabilization method according to claim 1 is characterized in that, also is provided with behind described the 3rd estimation and the compensation process:

The moving target extraction step is based on the original image motion smoothing of described current frame image and the result after the compensation, with respect to the original image detection moving target of reference frame image.

3. electronic image stabilization method according to claim 2 is characterized in that, in the described stratification step, and described intermediate layer image and be positioned at top and image that resolution is minimum and obtain based on the mode of mean filter.

4. electronic image stabilization method according to claim 3 is characterized in that, in described the second estimation and the compensation process, utilizes 2 times of described the first translational motion vector that described intermediate layer image is compensated.

5. the electronic steady image system for moving object detection under the DE Camera Shake is characterized in that, comprising:

Hierarchical block, original image to reference frame image and current frame image carries out respectively Gaussian Blur and reduces resolution, be respectively described reference frame image and current frame image and determine three tomographic images, comprising: be positioned at the highest original image of lowermost layer and resolution, intermediate layer image and be positioned at image top and that resolution is minimum;

The first motion estimation module, be used for based on piecemeal Gray Projection method, determine described be arranged in top and current frame image that resolution is minimum with respect to being positioned at each corresponding motion vector in piece zone of reference frame image top and that resolution is minimum, and then obtain a plurality of motion vectors; In described a plurality of motion vectors, the regional corresponding motion vector of moving target may appear in removal, according to remaining motion vector computation the first translational motion vector;

The second estimation and compensating module are used for based on described the first translational motion vector that obtains, the intermediate layer image of compensation current frame image; In the intermediate layer image of the described current frame image after compensation, remove the described zone that moving target may occur, the present frame intermediate layer image after obtaining to compensate; Utilize BMA, obtain present frame intermediate layer image after the described compensation with respect to the second motion translation vector of reference frame intermediate layer image;

The 3rd estimation and compensating module based on the first translational motion vector and the second motion translation vector, compensate the described original image that is positioned at the highest current frame image of lowermost layer and resolution, obtain the present frame original image after the compensation; Utilize the characteristic matching method that the original image of reference frame image and the present frame original image after the described compensation are carried out motion smoothing and compensation.

6. electronic steady image according to claim 5 system is characterized in that, also is connected with behind described the 3rd estimation and the compensating module:

The moving target extraction module is used for based on the original image motion smoothing of described current frame image and the result after the compensation, with respect to the original image detection moving target of reference frame image.

7. electronic steady image according to claim 6 system is characterized in that, in the described hierarchical block, and described intermediate layer image and be positioned at top and image that resolution is minimum and obtain based on the mode of mean filter.

8. electronic steady image according to claim 7 system is characterized in that, in described the second estimation and the compensating module, utilizes 2 times of described the first translational motion vector that described intermediate layer image is compensated.

Images