CN102098440A - Electronic image stabilizing method and electronic image stabilizing system aiming at moving object detection under camera shake - Google Patents

Abstract

The invention discloses an electronic image stabilizing method and an electronic image stabilizing system aiming at moving object detection under camera shake. The method comprises the following steps: carrying out guassian blur on the original image, lowering the resolution ratio and determining three layers of images; determining a first translational motion vector corresponding to an image with the lowest resolution ratio positioned at the highest layer based on a blocking gray projection method; compensating an interlayer image based on the acquired first translational motion vector; determining a second translational motion vector corresponding to the image positioned in the interlayer based on a block match algorithm; compensating the original image with the highest resolution ratio positioned at the lowest layer based on the first translational motion vector and the second translational motion vector; and smoothing and compensating the motion by utilizing a feature matching method. The method and system provided by the invention have good robustness to the dynamic background, the noise, and the consistency of motion, rotation and background of large-sized images, have superiority on the image quality, the visual effect and the calculation speed, and satisfy the requirements of the moving object detection under the camera shake after the image is stabilized.

Description

Electronic steady image method and system at moving object detection under the DE Camera Shake

Technical field

The present invention relates to the moving object detection under the dynamic scene image sequence, relate in particular to a kind of electronic steady image method and system at moving object detection under the DE Camera Shake.

Background technology

Along with the progress and the development of science and technology of human society, utilize computer to realize that human visual performance becomes in the present computer realm one of popular topic.The research of Detection for Moving Target is an important subject of computer vision field.Moving object detection is exactly that the technology such as image sequence utilization Digital Signal Processing that comprise movable information are suitably handled, detect and extract the prospect that has relative motion in the image sequence with background, determine the position of moving target, it has all had in many fields such as video monitoring, virtual reality, robot navigation, military aiming, tv edit, medical image analysis widely uses, and therefore has important use value and vast potential for future development.

But because the occasion of various Video Applications is not quite similar, residing environment of moving target and background are ever-changing, and this adaptability and robustness to the moving object detection algorithm is had higher requirement.Wherein, the moving object detection under the DE Camera Shake is the difficult point and the focus of field of video image processing research, also becomes a big obstacle of video image processing system practicality and reliability day by day.DE Camera Shake is meant because skew or the rotation that the camera lens that factors such as wind, exterior vibration cause takes place.DE Camera Shake can cause generation skew or rotation between adjacent image, causes image pixel coordinate correspondence one by one.

Summary of the invention

The objective of the invention is to needs of problems at moving object detection under the camera shake, take into full account the moving target in the scene removes dither algorithm to video camera influence, a kind of electronic steady image method and system at moving object detection under the DE Camera Shake is provided, based on the present invention, the consistency of dynamic background, noise, large scale image motion, rotation and the background that occurs in can the video to detection process of moving target has good robust property.

A kind of electronic steady image method of the present invention at moving object detection under the DE Camera Shake, comprise the steps: stratification step, original image to reference frame image and current frame image carries out Gaussian Blur respectively and reduces resolution, be respectively described reference frame image and current frame image and determine three tomographic images, comprising: be positioned at the highest original image of lowermost layer and resolution, intermediate layer image and be positioned at image top and that resolution is minimum; First motion-estimation step, based on piecemeal gray scale sciagraphy, determine described each piece zone that is arranged in top and the current frame image that resolution is minimum with respect to being positioned at each piece area relative motion vector top and reference frame image that resolution is minimum, and then obtain a plurality of motion vectors; In described a plurality of motion vectors, the area relative motion vector of moving target may appear in removal, according to remaining motion vector computation first translational motion vector; Second estimation and compensation process, based on described first translational motion vector that obtains, the intermediate layer image of compensation current frame image; In the intermediate layer image of the described current frame image after compensation, remove the described zone that moving target may occur, the present frame intermediate layer image after obtaining to compensate; Utilize BMA, obtain present frame intermediate layer image after the described compensation with respect to the second motion translation vector of reference frame intermediate layer image; The 3rd estimation and compensation process based on first translational motion vector and the second motion translation vector, compensate the described original image that is positioned at the highest current frame image of lowermost layer and resolution, obtain the present frame original image after the compensation; Utilize the characteristic matching method that the original image of reference frame image and the present frame original image after the described compensation are carried out motion smoothing and compensation.

Above-mentioned electronic steady image method, also be provided with behind preferred described the 3rd estimation and the compensation process: the moving target extraction step, based on the original image motion smoothing of described current frame image and the result after the compensation, with respect to the original image detection moving target of reference frame image.

Above-mentioned electronic steady image method, in the preferred described stratification step, described intermediate layer image and be positioned at top and image that resolution is minimum and obtain based on the mode of mean filter.

Above-mentioned electronic steady image method in preferred described second estimation and the compensation process, utilizes 2 times of described first translational motion vector that described intermediate layer image is compensated.

Above-mentioned electronic steady image method, in preferred described the 3rd estimation and the compensation process, based on affine model, carry out motion smoothing and compensation, wherein, the parameter of affine model is determined in the following way: original image and the present frame original image after the described compensation to reference frame image carry out yardstick invariant features converting characteristic coupling, find out corresponding points, obtain kinematic parameter in the substitution affine model, realize the original image translation of current frame image and the estimation that rotatablely moves; Based on the parameter that estimation draws, carry out level and smooth and compensation to the present frame original image sequence after the described compensation; Wherein, affine model is determined by following formula:

$\left(\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\end{array}\right)=\left(\begin{array}{cc}{a}_1& a_2\\ a_3& a_4\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)+\left(\begin{array}{c}{d}_1\\ {\mathrm{<figure-callout\; id="2"\; label="d"\; filenames="HDA0000038700550000031.png"\; state="\{\{state\}\}">d</figure-callout>}}_2\end{array}\right),$

Wherein A certain pixel is at the coordinate of reference frame and present frame, d in the difference presentation video ₁, d ₂Expression

With respect to

Side-play amount at x axle and y axle.a ₁～a ₄Expression rotation parameter and convergent-divergent among a small circle.

On the other hand, the invention also discloses a kind of electronic steady image system, comprising: hierarchical block, first motion estimation module, second estimation and compensating module and the 3rd estimation and compensating module at moving object detection under the DE Camera Shake.Wherein:

Hierarchical block is used for the original image of reference frame image and current frame image is carried out Gaussian Blur respectively and reduces resolution, be respectively described reference frame image and current frame image and determine three tomographic images, comprising: be positioned at the highest original image of lowermost layer and resolution, intermediate layer image and be positioned at image top and that resolution is minimum; First motion estimation module is used for based on piecemeal gray scale sciagraphy, determine described be arranged in top and current frame image that resolution is minimum with respect to being positioned at each piece area relative motion vector of reference frame image top and that resolution is minimum, and then obtain a plurality of motion vectors; In described a plurality of motion vectors, the area relative motion vector of moving target may appear in removal, according to remaining motion vector computation first translational motion vector; Second estimation and compensating module are used for based on described first translational motion vector that obtains, the intermediate layer image of compensation current frame image; In the intermediate layer image of the described current frame image after compensation, remove the described zone that moving target may occur, the present frame intermediate layer image after obtaining to compensate; Utilize BMA, obtain present frame intermediate layer image after the described compensation with respect to the second motion translation vector of reference frame intermediate layer image; The 3rd estimation and compensating module be based on first translational motion vector and the second motion translation vector, compensates the described original image that is positioned at the highest current frame image of lowermost layer and resolution, obtains the present frame original image after the compensation; Utilize the characteristic matching method that the original image of reference frame image and the present frame original image after the described compensation are carried out motion smoothing and compensation.

Above-mentioned electronic steady image system, also be connected with behind preferred described the 3rd estimation and the compensating module: the moving target extraction module, be used for based on the original image motion smoothing of described current frame image and the result after the compensation, with respect to the original image detection moving target of reference frame image.

Above-mentioned electronic steady image system, in the preferred described hierarchical block, described intermediate layer image and be positioned at top and image that resolution is minimum and obtain based on the mode of mean filter.

Above-mentioned electronic steady image system in preferred described second estimation and the compensating module, utilizes 2 times of described first translational motion vector that described intermediate layer image is compensated.

Above-mentioned electronic steady image system, in preferred described the 3rd estimation and the compensating module, based on affine model, carry out motion smoothing and compensation, wherein, the parameter of affine model is determined in the following way: original image and the present frame original image after the described compensation to reference frame image carry out yardstick invariant features converting characteristic coupling, find out corresponding points, obtain kinematic parameter in the substitution affine model, realize the original image translation of current frame image and the estimation that rotatablely moves; Based on the parameter that estimation draws, carry out level and smooth and compensation to the present frame original image sequence after the described compensation; Wherein, affine model is determined by following formula:

$\left(\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\end{array}\right)=\left(\begin{array}{cc}{a}_1& a_2\\ a_3& a_4\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)+\left(\begin{array}{c}{d}_1\\ {\mathrm{<figure-callout\; id="2"\; label="d"\; filenames="HDA0000038700550000031.png"\; state="\{\{state\}\}">d</figure-callout>}}_2\end{array}\right),$

Wherein

A certain pixel is at the coordinate of reference frame and present frame, d in the difference presentation video ₁, d ₂Expression

With respect to

Side-play amount at x axle and y axle.a ₁～a ₄Expression rotation parameter and convergent-divergent among a small circle.

In terms of existing technologies, the present invention is by adopting the multiresolution layered approach, divide three layers image carried out different estimation and compensation method, solved a lot of interference problem in the steady picture process: characteristic matching method and piecemeal gray scale sciagraphy are relatively more responsive to change of background such as dynamic background, noises; The block-based histogram feature of BMA adapts to problems such as dynamic background, noise, but piece coupling computation complexity height is influenced by the background area consistency; Piecemeal gray scale sciagraphy estimated motion vector has been handled the large scale image motion, and has avoided the influence of moving target, but to the change of background sensitivity, image reduces resolution and suppressed local faint change of background.Piecemeal gray scale sciagraphy and BMA all can not be handled the rotation of image, utilize the rotating vector of characteristic matching method estimated image.

The present invention can both have good robust property for the consistency of the dynamic background that occurs in the video, noise, large scale image motion, rotation and background.And with respect to other algorithms, certain superiority is arranged all on picture quality, visual effect, computational speed, can satisfy the camera shake actual demand of moving object detection down behind the steady picture.

Description of drawings

Fig. 1 is the flow chart of steps that the present invention is directed to the electronic steady image method embodiment of moving object detection under the DE Camera Shake;

Fig. 2 is the multiresolution hierarchical diagram schematic diagram of image;

Fig. 3 is a layering multiresolution overall motion estimation vector flow graph schematic diagram;

Fig. 4 obtains the motion vector schematic diagram for the low-resolution image piecemeal utilizes sciagraphy;

Fig. 5 carries out electronic steady image foreground detection afterwards figure as a result;

Fig. 6 compares schematic diagram for rectification and the target detection that migrated image takes place;

Fig. 7 is that the Y-PSNR of sequence image compares schematic diagram;

Fig. 8 is the structured flowchart that the present invention is directed to the electronic steady image system embodiment of moving object detection under the DE Camera Shake;

Fig. 9 is the structured flowchart that the present invention is directed to the electronic steady image system embodiment of moving object detection under the DE Camera Shake.

Embodiment

For above-mentioned purpose of the present invention, feature and advantage can be become apparent more, the present invention is further detailed explanation below in conjunction with the drawings and specific embodiments.

With reference to Fig. 1, Fig. 1 is the flow chart of steps that the present invention is directed to the electronic steady image method embodiment of moving object detection under the DE Camera Shake, comprise the steps: stratification step S110, original image to reference frame image and current frame image carries out Gaussian Blur respectively and reduces resolution, be respectively three tomographic images of determining of reference frame image and current frame image, comprise: be positioned at the highest original image of lowermost layer and resolution, intermediate layer image and be positioned at image top and that resolution is minimum; The first motion-estimation step S120, based on piecemeal gray scale sciagraphy, determine to be arranged in top and current frame image that resolution is minimum with respect to each piece area relative motion vector of reference frame image, and then obtain a plurality of motion vectors; In a plurality of motion vectors, the area relative motion vector of moving target may appear in removal, according to remaining motion vector computation first translational motion vector; Second estimation and compensation process S130, based on first translational motion vector that obtains, the intermediate layer image of compensation current frame image; In the intermediate layer image of the current frame image after compensation, the zone of moving target may appear in removal, for remaining zone, utilize BMA to be compensated and remove the intermediate layer current frame image that motion target area may occur the second motion translation vector with respect to reference frame image; The 3rd estimation and compensation process S140 based on first translational motion vector and the second motion translation vector, compensate the described original image that is positioned at the highest current frame image of lowermost layer and resolution, obtain the present frame original image after the compensation; Utilize the characteristic matching method that the original image of reference frame image and the present frame original image after the described compensation are carried out motion smoothing and compensation.

Present embodiment all has good robust property for the consistency of the dynamic background that occurs in the video, noise, large scale image motion, rotation and background; And with respect to other algorithms, certain superiority is arranged all on picture quality, visual effect, computational speed, can satisfy the camera shake actual demand of moving object detection down behind the steady picture.

The invention will be further described below in conjunction with the drawings and specific embodiments.

The present invention carries out image Gaussian Blur and reduces resolution, is divided into three layers, and three layers of image in different resolution are carried out different processing respectively.Adopt gray scale sciagraphy, BMA, improving one's methods of characteristic matching method that each tomographic image is carried out estimation and compensation respectively.Simply introduce these three kinds of methods below respectively.

1, gray scale sciagraphy

The gray scale sciagraphy is to utilize the ranks intensity profile to carry out the motion estimation algorithm of related operation.This algorithm utilizes the ranks gray scale drop shadow curve of image to do the motion vector that related operation obtains image.Algorithm mainly is made up of image mapped and related operation.

1) image mapped

After using histogram equalization to carry out preliminary treatment to the two dimensional image of each frame input, be mapped to two independently one dimension waveforms.Projecting method can be described by formula (3) (4),

$G_K\left(i\right)=\underset{j}{\mathrm{\&Sigma;}}{G}_k(i,j)---\left(3\right)$

$G_K\left(j\right)=\underset{i}{\mathrm{\&Sigma;}}{G}_k(i,j)---\left(4\right)$

G in the formula _K(i), G _K(j) be respectively that k two field picture i is capable, the gray value sum of each pixel of j row, G _K(i j) is (i, j) grey scale pixel value of position on the k two field picture.

2) correlation computations

The projection waveform of k two field picture projection waveform and reference picture is done cross-correlation calculation.Wherein the row correlation computations can be described by formula (5),

$C\left(w\right)={\mathrm{\&Sigma;}}_{j=1}^N{[{\mathrm{Col}}_k(j+w)-{\mathrm{Col}}_r(M+j)]}^2---\left(5\right)$

In the formula, 1≤w≤2M+1, Col _k(j), Col _r(j) be respectively the gray scale projection value of the j row of k two field picture and reference frame image, N is the length of row, and M is that displacement vector is with respect to the search width of reference frame in a side.

2, block matching algorithm

Based on the estimation of piece coupling, be meant picture frame is divided into the identical a series of sub-piece of size that each sub-block size is N * N.For a sub-piece in the reference frame, in the k frame, seek the most similar sub-piece identical with its size, the matching criterior commonly used of seeking match block has three kinds, i.e. minimum average B configuration absolute difference (MAD), least mean-square error (MSE) and Normalized Cross Correlation Function (NCCF).This paper adopts least mean-square error (MSE) criterion to mate, can describe by formula (6),

$\mathrm{MSE}\left(d\right)=\frac{1}{N_x\&times;N_y}\underset{x=1}{\overset{N_x}{\mathrm{\&Sigma;}}}\underset{y=1}{\overset{N_y}{\mathrm{\&Sigma;}}}{[I_k(x+d_x,y+d_y)-I_t(x,y)]}^2---\left(6\right)$

Wherein, I _k(x, y), I _t(x, y) expression k, t two field picture mid point (x, the gray value of y) locating, d=(d _x, d _y) be translation parameters.

3, characteristic matching method

Based on the motion estimation algorithm of characteristic matching selected characteristic in reference frame at first, in subsequent frame, feature is positioned then, utilize the feature set corresponding relation at last, ask for the global motion parameter.The algorithm basic step is described below:

A) feature detection: a certain two field picture in the image sequence is handled, extracted characteristics of image, number of features is m.

B) characteristic matching: the spatial domain relation between use characteristic identifier, similarity measurement and feature, determine the positional information of feature at current frame image.

C) transformation model is estimated: the correspondence position of characteristic quantity concerns simultaneous equations in the substitution motion model, separates the overdetermined equation group, obtains global motion vector.

Below in conjunction with a concrete example, the present invention realizes that the method for moving object detection is described in detail to foundation:

Step 1: stratification step

In order to reduce the influence that image local motion and moving target move, the thought of image pyramid layering is introduced in invention, divides three layers to reduce resolution to original image.Because low resolution image can the smoothed image local motion, reduce noise effect, improve the algorithm real-time, in low-resolution image, the low-angle image rotation that DE Camera Shake causes can be left in the basket, and the image overall motion can be reduced to translational motion.As shown in Figure 2, lowermost layer is an original image, reduces resolution from bottom gradually to high level, generally uses simple mean filter, can be described by formula (7):

$f_k^{(l+1)}(i,j)=\frac{1}{4}\underset{m=0}{\overset{1}{\mathrm{\&Sigma;}}}\underset{n=0}{\overset{1}{\mathrm{\&Sigma;}}}{f}_k^{\left(l\right)}(2i+m,2j+n)1=\mathrm{0,1},...N---\left(7\right)$

In the formula (7), (1+1) layer of representing the k frame is at (i, j) pixel value of position.

K two field picture for input.

Step 2: first motion-estimation step and second estimation and compensation process

This step is low-resolution image estimation and compensation.Based on the image three-decker, Fig. 3 has provided the flow chart based on the image overall estimation of three-decker.Invention will below be the specific implementation of method at three tomographic images to doing concrete elaboration in the top algorithm block diagram:

1. the 3rd layer of low-resolution image carried out piecemeal, then each piecemeal is utilized sciagraphy, estimate the motion vector of each sub-piece.As shown in Figure 4, A, B figure is respectively the image of reference frame and present frame among the figure, certain skew has taken place with respect to reference frame image in present frame, C figure has represented to utilize the motion vector of the subregion that piecemeal gray scale sciagraphy obtains, wherein because the existence of dolly has caused the vector of dolly place subregion estimated movement vector and other subregions estimation to take place than large deviation.Each sub regions motion vector is added and average, the global motion vector MV (3) of formation, the adding and process of vector, eliminated probability of occurrence little and with the bigger vector of consistent motion vector difference.

2. the 2nd layer current frame image is carried out the global motion compensation of 2 * MV (3), and image is carried out piecemeal, partitioned mode is similar to the 3rd layer, and the number of piece is identical, but size is twice, but removes the influence that the edge causes in the reality, and size is smaller slightly.Next find out and the corresponding subregion of the 3rd straton piece at the 2nd layer, these several sub regions are carried out piece coupling, ask for motion vector and add and on average obtain motion vector MV (2).Utilize the motion estimation vectors MV (2) of the 2nd tomographic image that vector MV (3) is had the compensating action of refining.

Step 3: the 3rd estimation and compensation process

This step is original image estimation and compensation.At first utilize vector 4 * MV (3) and 2 * MV (2) to compensate to the 1st layer, because MV (3) and MV (2) vector have just reacted the translational motion of image, algorithm will add the estimation to the image rotation parameter in 1 layer estimation, algorithm utilizes yardstick invariant features conversion (SIFT) characteristic matching to find out corresponding points, obtain kinematic parameter in the substitution affine model, realize image translation and the estimation that rotatablely moves.We choose n to angular coordinate, in coordinate substitution affine model, can get the linear system equation:

N then can obtain (2n * 6) the rank system of linear equations on the real number field to matching characteristic, and solution of equations is the global motion parameter of interframe.And equation number 2n＞6 that obtained generally speaking, i.e. overdetermined equation group.This equation group does not have separating under the ordinary meaning, for this reason the demand equation group least square solution.Definition energy function ρ _x, ρ _y:

${\mathrm{\&rho;}}_x=\underset{i=0}{\overset{N}{\mathrm{\&Sigma;}}}{(a_1{x}_i+a_2{y}_i+d_1-x^{\&prime;})}^2---\left(9\right)$

${\mathrm{\&rho;}}_y=\underset{i=0}{\overset{N}{\mathrm{\&Sigma;}}}{(a_3{x}_i+a_4{y}_i+d_2-y^{\&prime;})}^2---\left(10\right)$

Then have:

$\frac{\&PartialD;{\mathrm{\&rho;}}_x}{\&PartialD;a_1}=\frac{{\&PartialD;\mathrm{\&rho;}}_x}{\&PartialD;a_2}=\frac{\&PartialD;{\mathrm{\&rho;}}_x}{\&PartialD;{\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="HDA0000038700550000042.png"\; state="\{\{state\}\}">d</figure-callout>}}_1}=\frac{\&PartialD;{\mathrm{\&rho;}}_y}{\&PartialD;a_3}=\frac{\&PartialD;{\mathrm{\&rho;}}_y}{\&PartialD;a_4}=\frac{\&PartialD;{\mathrm{\&rho;}}_y}{\&PartialD;{\mathrm{<figure-callout\; id="2"\; label="d"\; filenames="HDA0000038700550000031.png"\; state="\{\{state\}\}">d</figure-callout>}}_2}=0---\left(11\right)$

Thereby can try to achieve parameter a ₁～a ₄, d ₁～d ₂, draw parameter a by estimation ₁～a ₄, d ₁～d ₂After, the substitution affine model can compensate image sequence, can be described by formula (12) (13).

$x=\frac{{a_4}^{*}{x}^{,}-{a_2}^{*}{y}^{,}-{{\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="HDA0000038700550000042.png"\; state="\{\{state\}\}">d</figure-callout>}}_1}^{*}{a}_4+{a_2}^{*}{d}_2}{{a_1}^{*}{a}_4-{a_2}^{*}{a}_3}---\left(12\right)$

$y=\frac{{a_3}^{*}{x}^{,}-{a_1}^{*}{y}^{,}-{{\mathrm{<figure-callout\; id="1"\; label="d"\; filenames="HDA0000038700550000042.png"\; state="\{\{state\}\}">d</figure-callout>}}_1}^{*}{a}_3+{a_1}^{*}{d}_2}{{a_2}^{*}{a}_3-{a_1}^{*}{a}_4}---\left(13\right)$

In the formula

A certain pixel is at the coordinate of reference frame and present frame, d in the difference presentation video ₁, d ₂Expression With respect to

Side-play amount at x axle and y axle.a ₁～a ₄Expression rotation parameter and convergent-divergent among a small circle.

Step 4: moving target extracts

Invention is adopted and is carried out the extraction of moving target based on the method for background subtraction, verifies steady picture effect.At first current frame image and background image are carried out calculus of differences, then difference image is carried out the binaryzation operation, if pixel value is greater than a certain threshold value in the difference image, judge that then this pixel belongs to motion target area, otherwise, just judge that this pixel belongs to the background area, target detection result as shown in Figure 5.

It is carrier that the present invention adopts 320 * 240 image sequence, and a large amount of experiments show, our method is for the moving object detection under the DE Camera Shake of indoor and outdoor, can reach surely to look like the result preferably.As shown in Figure 6, considered the influence that moving target removes dither algorithm to video camera, the zone that may have moving target removed that the result of detection is better than the result that the three kinds of algorithms in front are corrected, and wherein also contains the rotation of image among the 3rd width of cloth figure.Among Fig. 6, A figure expression reference frame image, B figure expression current frame image, C figure is that the gray scale sciagraphy is corrected the back image and A desires to make money or profit with the result of calculus of finite differences detection, D figure is that BMA is corrected the back image and A desires to make money or profit with the result of calculus of finite differences detection, E figure is that yardstick invariant features converting characteristic matching method is corrected the back image and A desires to make money or profit with the result of calculus of finite differences detection, the three kinds of method testing results in front detect wrong more as can be seen, F figure is that our algorithm is corrected the back image and A desires to make money or profit with the result of calculus of finite differences detection, detects wrong less in the testing result.By the result as can be seen, only use first three methods not consider the influence of moving target, caused algorithm to estimate kinematic parameter generation deviation algorithm.

Fig. 7 has provided the Y-PSNR value of 5 two field pictures in the sequence image.Y-PSNR is a kind of assessment of performance standard, and is identical with mean square deviation in essence, its reflection be Y-PSNR between reference frame image and the current frame image, the Y-PSNR value is high more, the video stabilization effect is good more.Y-PSNR is defined as follows:

$\mathrm{PNSR}=10\log \frac{{255}^2}{\mathrm{MSE}(I_k,I_t)}---\left(14\right)$

MSE (I wherein _k, I _t) mean square deviation, be defined as follows:

$\mathrm{MSE}(I_k,I_t)=\frac{1}{M\&times;N}\underset{x=1}{\overset{M}{\mathrm{\&Sigma;}}}\underset{Y=1}{\overset{N}{\mathrm{\&Sigma;}}}{[I_k(x,y)-I_t(x,y)]}^2---\left(15\right)$

From above result as can be seen, the method that the present invention proposes has been considered the existence of moving target, can effectively handle the video of the DE Camera Shake that contains moving target, and can high-qualityly carry out electronic steady image.Consistency for the dynamic background that occurs in the video, noise, large scale image motion, rotation and background can both have good robust property.With respect to other algorithms, certain superiority is all arranged on picture quality, visual effect, computational speed, this scheme computing is simple and convenient, the reliability height, real-time is good, can satisfy the camera shake actual demand of moving object detection down.

Other method the invention also discloses a kind of electronic steady image system at moving object detection under the DE Camera Shake.With reference to Fig. 8, Fig. 8 is the structured flowchart that the present invention is directed to the electronic steady image system embodiment of moving object detection under the DE Camera Shake, comprising: stratification step 80, first motion-estimation step 82, second estimation and compensation process 84 and the 3rd estimation and compensation process 86.

Wherein: hierarchical block 80 is used for the original image of reference frame image and current frame image is carried out Gaussian Blur respectively and reduces resolution, be respectively three tomographic images of determining of described reference frame image and current frame image, comprise: be positioned at the highest original image of lowermost layer and resolution, intermediate layer image and be positioned at image top and that resolution is minimum; First motion estimation module 82 is used for based on piecemeal gray scale sciagraphy, determine described be arranged in top and current frame image that resolution is minimum with respect to being positioned at each piece area relative motion vector of reference frame image top and that resolution is minimum, and then obtain a plurality of motion vectors; In described a plurality of motion vectors, the area relative motion vector of moving target may appear in removal, according to remaining motion vector computation first translational motion vector; Second estimation and compensating module 84 are used for based on first translational motion vector that obtains, the intermediate layer image of compensation current frame image; In the intermediate layer image of the current frame image after compensation, the zone of moving target may appear in removal, obtains the first present frame intermediate layer image; Zone for remaining utilizes BMA, obtains the second motion translation vector of the first present frame intermediate layer image with respect to reference frame intermediate layer image; The 3rd estimation and compensating module 86 be based on first translational motion vector and the second motion translation vector, compensates the described original image that is positioned at the highest current frame image of lowermost layer and resolution, obtains the present frame original image after the compensation; Utilize the characteristic matching method that the original image of reference frame image and the present frame original image after the described compensation are carried out motion smoothing and compensation.

With reference to Fig. 9, Fig. 9 is the structured flowchart at another embodiment of electronic steady image system of moving object detection under the DE Camera Shake.This embodiment also is connected with moving target extraction module 88 on the basis of Fig. 8, this module is used for detecting moving target based on the result after original image motion smoothing and the compensation.

In one embodiment, in second estimation and the compensating module 84, utilize 2 times of first translational motion vector that described intermediate layer image is compensated.

In one embodiment, in the 3rd estimation and the compensating module 86, comprise also and obtain the affine model parameter acquiring unit that this unit is used for:

Present frame original image after described reference frame original image and the described compensation is carried out yardstick invariant features converting characteristic coupling, find out corresponding points, obtain kinematic parameter in the substitution affine model, realize current frame image translation and the estimation that rotatablely moves; Based on the parameter that estimation draws, carry out level and smooth and compensation to the original image sequence; Wherein, affine model is determined by following formula:

$\left(\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\end{array}\right)=\left(\begin{array}{cc}{a}_1& a_2\\ a_3& a_4\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)+\left(\begin{array}{c}{d}_1\\ {\mathrm{<figure-callout\; id="2"\; label="d"\; filenames="HDA0000038700550000031.png"\; state="\{\{state\}\}">d</figure-callout>}}_2\end{array}\right),$

Wherein

A certain pixel is at the coordinate of reference frame and present frame, d in the difference presentation video ₁, d ₂Expression With respect to

Side-play amount at x axle and y axle.a ₁～a ₄Expression rotation parameter and convergent-divergent among a small circle.

More than a kind of electronic steady image method and system at moving object detection under the DE Camera Shake provided by the present invention described in detail, used specific embodiment herein principle of the present invention and execution mode are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, part in specific embodiments and applications all can change.In sum, this description should not be construed as limitation of the present invention.

Claims (10)

1. the electronic steady image method at moving object detection under the DE Camera Shake is characterized in that, comprises the steps:

Stratification step, original image to reference frame image and current frame image carries out Gaussian Blur respectively and reduces resolution, be respectively described reference frame image and current frame image and determine three tomographic images, comprising: be positioned at the highest original image of lowermost layer and resolution, intermediate layer image and be positioned at image top and that resolution is minimum;

First motion-estimation step, based on piecemeal gray scale sciagraphy, determine described each piece zone that is arranged in top and the current frame image that resolution is minimum with respect to being positioned at each piece area relative motion vector top and reference frame image that resolution is minimum, and then obtain a plurality of motion vectors; In described a plurality of motion vectors, the area relative motion vector of moving target may appear in removal, according to remaining motion vector computation first translational motion vector;

Second estimation and compensation process, based on described first translational motion vector that obtains, the intermediate layer image of compensation current frame image; In the intermediate layer image of the described current frame image after compensation, remove the described zone that moving target may occur, the present frame intermediate layer image after obtaining to compensate; Utilize BMA, obtain present frame intermediate layer image after the described compensation with respect to the second motion translation vector of reference frame intermediate layer image;

The 3rd estimation and compensation process based on first translational motion vector and the second motion translation vector, compensate the described original image that is positioned at the highest current frame image of lowermost layer and resolution, obtain the present frame original image after the compensation; Utilize the characteristic matching method that the original image of reference frame image and the present frame original image after the described compensation are carried out motion smoothing and compensation.

2. electronic steady image method according to claim 1 is characterized in that, also is provided with behind described the 3rd estimation and the compensation process:

The moving target extraction step is based on the original image motion smoothing of described current frame image and the result after the compensation, with respect to the original image detection moving target of reference frame image.

3. electronic steady image method according to claim 2 is characterized in that, in the described stratification step, and described intermediate layer image and be positioned at top and image that resolution is minimum and obtain based on the mode of mean filter.

4. electronic steady image method according to claim 3 is characterized in that, in described second estimation and the compensation process, utilizes 2 times of described first translational motion vector that described intermediate layer image is compensated.

5. electronic steady image method according to claim 4 is characterized in that, in described the 3rd estimation and the compensation process, based on affine model, carries out motion smoothing and compensation, and wherein, the parameter of affine model is determined in the following way:

Original image and the present frame original image after the described compensation to reference frame image carry out yardstick invariant features converting characteristic coupling, find out corresponding points, obtain kinematic parameter in the substitution affine model, realize the original image translation of current frame image and the estimation that rotatablely moves;

Based on the parameter that estimation draws, carry out level and smooth and compensation to the present frame original image sequence after the described compensation; Wherein, affine model is determined by following formula:

$\left(\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\end{array}\right)=\left(\begin{array}{cc}{a}_1& a_2\\ a_3& a_4\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)+\left(\begin{array}{c}{d}_1\\ d_2\end{array}\right),$

Wherein

A certain pixel is at the coordinate of reference frame and present frame, d in the difference presentation video ₁, d ₂Expression

With respect to

Side-play amount at x axle and y axle.a ₁～a ₄Expression rotation parameter and convergent-divergent among a small circle.

6. the electronic steady image system at moving object detection under the DE Camera Shake is characterized in that, comprising:

Hierarchical block, original image to reference frame image and current frame image carries out Gaussian Blur respectively and reduces resolution, be respectively described reference frame image and current frame image and determine three tomographic images, comprising: be positioned at the highest original image of lowermost layer and resolution, intermediate layer image and be positioned at image top and that resolution is minimum;

First motion estimation module, be used for based on piecemeal gray scale sciagraphy, determine described be arranged in top and current frame image that resolution is minimum with respect to being positioned at each piece area relative motion vector of reference frame image top and that resolution is minimum, and then obtain a plurality of motion vectors; In described a plurality of motion vectors, the area relative motion vector of moving target may appear in removal, according to remaining motion vector computation first translational motion vector;

Second estimation and compensating module are used for based on described first translational motion vector that obtains, the intermediate layer image of compensation current frame image; In the intermediate layer image of the described current frame image after compensation, remove the described zone that moving target may occur, the present frame intermediate layer image after obtaining to compensate; Utilize BMA, obtain present frame intermediate layer image after the described compensation with respect to the second motion translation vector of reference frame intermediate layer image;

The 3rd estimation and compensating module based on first translational motion vector and the second motion translation vector, compensate the described original image that is positioned at the highest current frame image of lowermost layer and resolution, obtain the present frame original image after the compensation; Utilize the characteristic matching method that the original image of reference frame image and the present frame original image after the described compensation are carried out motion smoothing and compensation.

7. electronic steady image according to claim 6 system is characterized in that, also is connected with behind described the 3rd estimation and the compensating module:

The moving target extraction module is used for based on the original image motion smoothing of described current frame image and the result after the compensation, with respect to the original image detection moving target of reference frame image.

8. electronic steady image according to claim 7 system is characterized in that, in the described hierarchical block, and described intermediate layer image and be positioned at top and image that resolution is minimum and obtain based on the mode of mean filter.

9. electronic steady image according to claim 8 system is characterized in that, in described second estimation and the compensating module, utilizes 2 times of described first translational motion vector that described intermediate layer image is compensated.

10. electronic steady image according to claim 9 system is characterized in that, in described the 3rd estimation and the compensating module, based on affine model, carries out motion smoothing and compensation, and wherein, the parameter of affine model is determined in the following way:

Original image and the present frame original image after the described compensation to reference frame image carry out yardstick invariant features converting characteristic coupling, find out corresponding points, obtain kinematic parameter in the substitution affine model, realize the original image translation of current frame image and the estimation that rotatablely moves;

Based on the parameter that estimation draws, carry out level and smooth and compensation to the present frame original image sequence after the described compensation; Wherein, affine model is determined by following formula:

$\left(\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\end{array}\right)=\left(\begin{array}{cc}{a}_1& a_2\\ a_3& a_4\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right)+\left(\begin{array}{c}{d}_1\\ d_2\end{array}\right),$

Wherein

A certain pixel is at the coordinate of reference frame and present frame, d in the difference presentation video ₁, d ₂Expression

With respect to

Side-play amount at x axle and y axle.a ₁～a ₄Expression rotation parameter and convergent-divergent among a small circle.

Images