CN102831580B - Method for restoring image shot by cell phone based on motion detection - Google Patents

Abstract

The invention discloses a method for restoring an image shot by a cell phone based on motion detection. The problems that the texture of the prior art is rich, the structure is complicated and the image restoration under an obvious outline scene is incorrect are solved. The method comprises the following steps of: preparing a frame by utilizing shooting of a shutter frame in a short time before shooting; training a background model based on a hybrid gaussian background model to acquire the background model which is close to the real scene; performing outline extraction, false detection correction and motion target sequence extraction by a motion detection module to acquire a motion target sequence which is erased by a user; and finally, replacing pixel values at corresponding positions in a target by pixel values of a background model frame in a target restoration range selected by a user to complete the restoration of image information and acquire a shutter frame image which does not include motion obstacles. By the method, a target erasing area in the lower image of the scene is correctly restored, and the method is used for an intelligent cell phone to shoot pictures.

Description

Based on the image shot by cell phone restorative procedure that motion detects

Technical field

The invention belongs to video and technical field of image processing, relate to image repair, is a kind of by target detection technique, and a kind of method of the moving target swarmed into suddenly in erasing image, is mainly used in the reparation of smart mobile phone shooting image.

Background technology

Along with the widespread use of smart mobile phone, the camera function of user to smart mobile phone is had higher requirement.User, when taking pictures, because how numerous and diverse environment people is, often can swarm into uninvited guest in picture, make photo seem mixed and disorderly, and it is fuzzy to produce motion smear, affects the whole structure of photo.So need to carry out the erasing of by-end thing to this intruder.

Traditional by-end image wipe technology utilizes the related pixel information around erase area to estimate to fill the pixel value wiping out this position after by-end.Basic skills has following three kinds:

One, based on Partial Differential Equation method.This method is the marginal information utilizing region to be repaired, determines diffuse information and dispersal direction, anisotropically spreads in border from zone boundary.The method for the image mending of gap form, in the face of the repair efficiency of larger image block is not good.

Two, based on picture breakdown method.This method is that picture breakdown is become structure division and texture part, and structure division adopts primary image restore design to repair, and texture part adopts texture synthesis method to fill.Easily there is the aplastic situation of regional area in the method.

Three, based on the texture synthesis method of sample.This method chooses a pixel from the border in region to be repaired, by this point centered by, according to the textural characteristics of image, choose sizeable texture block, Texture Matching block close is with it found again around repairing area, carry out this texture block alternative, the method when outline line obviously and texture-rich be difficult to obtain good effect.

Three kinds of basic skills above all utilize single-frame images information to carry out image repair, and all there is the false situation of reparation, especially when texture-rich, can produce wrong texture information, make repairing effect undesirable.

Summary of the invention

The object of the invention is to the deficiency for above-mentioned prior art, propose a kind of image shot by cell phone restorative procedure detected based on motion, in quick deletion shutter frame image, the moving obstacle of non-shooting main body, repairs image information accurately and effectively, improves effect of taking pictures.

For achieving the above object, technical scheme of the present invention comprises the steps:

(1) use mixed Gaussian background modeling method, build the Gaussian Background model of mobile phone photographed scene, to mate with the difference of Gaussian Background model frame according to the present frame of image shot by cell phone frame sequence and carry out sport foreground detection, obtain motion foreground picture;

(2) detect the motion foreground picture obtained from sport foreground and extract moving target, generate and detect agglomerate list:

(2a) target outermost layer contour detecting is carried out:

(2a1) from the upper left corner in sport foreground image, by from left to right, top-down order finds out its new frontier point, if can not find, complete all target outermost layer contour detecting in foreground image, otherwise using this new frontier point as initial boundary point S, completed the outermost layer contour detecting of the target comprising frontier point S by Contour extraction, make this frontier point S be current border point D;

(2a2) record the coordinate of current border point D, according to the boundary chain code direction of current border point D, find its adjacent boundary point N;

(2a3) judge whether to find adjacent boundary point N, if can not find, then complete the outermost layer contour detecting of the target comprising frontier point S, return step (2a1); Otherwise judge whether frontier point N overlaps with initial boundary point S, if overlapped; would complete the outermost layer contour detecting of the target comprising frontier point S, return step (2a1), otherwise using N as current border point D, return step (2a2);

(2b) characteristic information of the target outermost layer profile detected is calculated:

According to the point coordinate information of each target outermost layer profile, obtain the centre of form of this target these three features of wide W and high H:

$\stackrel{\‾}{i}=\frac{\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{p}_n^{i}f(p_n^i,p_n^j)}{\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f(p_n^i,p_n^j)},$ $\stackrel{\‾}{j}=\frac{\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{p}_n^{j}f(p_n^i,p_n^j)}{\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f(p_n^i,p_n^j)},$

$W=4\×\sqrt{\frac{\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\left(p_n^i\right)}^{2}f(p_n^i,p_n^j)}{\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f(p_n^i,p_n^j)}-{\left[\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{p}_n^{i}f(p_n^i,p_n^j)\right]}^2},$

$H=4\×\sqrt{\frac{\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{\left(p_n^j\right)}^{2}f(p_n^i,p_n^j)}{\underset{n=1}{\overset{N}{\mathrm{\Σ}}}f(p_n^i,p_n^j)}-{\left[\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{p}_n^{j}f(p_n^i,p_n^j)\right]}^2},$

Wherein, with be respectively x-axis and the y-axis coordinate of the n-th point, be the grey scale pixel value of the n-th point, N is the pixel number of target outermost layer profile;

(2c) according to closing on principle, polymerization process is carried out to target according to the target outermost layer contour feature information obtained:

For any two targets that step (2a) obtains, as target A and target B, calculate its centre of form apart from Dis _aB:

${\mathrm{Dis}}_{\mathrm{AB}}=\sqrt{{({\stackrel{\‾}{i}}_A-{\stackrel{\‾}{i}}_B)}^2+{({\stackrel{\‾}{j}}_A-{\stackrel{\‾}{j}}_B)}^2},$

If then target A and target B polymerization is become target C, and delete target A and target B, the centre of form obtaining target C is wide W _c, high H _cbe respectively:

${\stackrel{\‾}{i}}_C=\frac{{\stackrel{\‾}{i}}_A+{\stackrel{\‾}{i}}_B}{2},$ ${\stackrel{\‾}{j}}_C=\frac{{\stackrel{\‾}{j}}_A+{\stackrel{\‾}{j}}_B}{2},$

$W_C=\frac{W_A+W_B}{2},$

$H_C=\frac{H_A+H_B}{2},$

Wherein, with be respectively x-axis and the y-axis coordinate of the target A centre of form, with be respectively x-axis and the y-axis coordinate of the target B centre of form, W _aand H _abe respectively the wide and high of target A, W _band H _bbe respectively the wide and high of target B, α _disfor the centre of form is apart from coefficient, its span is: 1.2≤α _dis≤ 1.4;

(2d) the outermost layer contour feature information of each target is described as lumpiness, and all lumpinesses that present frame obtains are arranged in order, generate and detect agglomerate list;

(3) utilize feature matching method to carry out the tracking of motion agglomerate to the agglomerate detected in agglomerate list, upgrade and follow the tracks of agglomerate list:

If it is empty for (3a) following the tracks of agglomerate list, then the detection agglomerate list of present frame is directly added to and follow the tracks of agglomerate list;

If (3b) following the tracks of agglomerate list is not sky, then travels through the detection agglomerate list of present frame and follow the tracks of agglomerate list, carrying out blob match by calculating the similarity detected in agglomerate list and the list of tracking agglomerate between each agglomerate;

(4) list of tracking agglomerate, sport foreground image and background model is utilized to complete the image information reparation of moving target.

Tool of the present invention has the following advantages:

1) the present invention is by the background model of photographed scene, utilizes the replacement of shutter frame image and background model pixel to reach reparation object, can accurately to texture-rich, complex structure and there is the target erasure region under the scene of apparent contour to repair.

2) the present invention carries out characteristic matching by the detection agglomerate list obtained and the list of tracking agglomerate photographed scene being carried out to Object Detecting and Tracking, to just detecting the major punishment periodical repair of agglomerate, can when moving target have shade and target static suddenly detect moving target exactly.

Accompanying drawing explanation

Fig. 1 is general flow chart of the present invention;

Fig. 2 is that the sport foreground in Fig. 1 detects sub-process figure;

Fig. 3 is the extraction moving target in Fig. 1, generates and detects agglomerate list sub-process figure;

Fig. 4 is that agglomerate list sub-process figure is followed the tracks of in the renewal in Fig. 1;

Fig. 5 is that the image information in Fig. 1 repairs sub-process figure;

Fig. 6 adopts the present invention to emulate the original image of use and the sport foreground image of extraction;

Fig. 7 is original image and the Contour extraction result figure of Contour extraction emulation;

Fig. 8 adopts the present invention to emulate the shutter frame of use and the result figure after repairing.

Specific implementation method

Referring to accompanying drawing, the present invention is described in further detail.

With reference to Fig. 1, specific implementation step of the present invention is as follows:

Step 1, uses mixed Gaussian background modeling method, builds the Gaussian Background model of mobile phone photographed scene.

Be the rule meeting gaussian random probability distribution according to scene background pixel value natural in the short time, and same pixel does not have the different pixel value of possibility in the same time, uses multiple average and standard deviation to characterize the background of a pixel; The present invention describes scene background by setting up M Gauss model to each pixel, and the value of M determines according to scene complexity, and value is 3 ~ 5.

The mathematical definition of mixture Gaussian background model is as follows:

Probability density function f (X _t) an available following M Gauss model function representation:

$f\left(X_t\right)=\underset{i=1}{\overset{M}{\mathrm{\Σ}}}{\mathrm{\ω}}_{i,t}\×\mathrm{\η}(X_t,{\mathrm{\μ}}_{i,t},{\mathrm{\Σ}}_{i,t})$

$\mathrm{\η}(X_t,{\mathrm{\μ}}_{i,t},{\mathrm{\Σ}}_{i,t})=\frac{1}{{\left(2\mathrm{\π}\right)}^{n/2}{\left|{\mathrm{\Σ}}_{i,t}\right|}^{1/2}}{e}^{-\frac{1}{2}{(X_t-{\mathrm{\μ}}_{i,t})}^T{\mathrm{\Σ}}_{i,t}^{-1}(X_t-{\mathrm{\μ}}_{i,t})}$

Wherein, η (X in above formula _t, μ _i,t, ∑ _i,t) be i-th Gaussian distribution of t, its average is μ _{i, t}, covariance matrix is ∑ _{i, t}, ω _{i, t}for the weights of each Gauss model, X _tfor t pixel value, n is X _tdimension.

According to above-mentioned principle, the concrete steps implementing the Gaussian Background model building mobile phone photographed scene are as follows:

(1a) M Gauss model is set up, by the average μ of first Gauss model to each pixel in video frame images ₀be initialized as the pixel value of the first two field picture, the average μ of all the other Gauss models _ibe initialized as random value, 0<i<M; All Gauss model standard deviation sigma are initialized as 10, by the weights omega of first Gauss model ₀be set to α is for upgrading coefficient, and its span is 0.001< α <0.01, the weights omega of all the other Gauss models _ibe set to

(1b) each frame in K frame, calculates the average μ of the color value v of this each pixel of two field picture each Gauss model corresponding with it _iabsolute difference △ _i=| v-μ _i|, to the average μ of each Gauss model _i, standard deviation sigma _iand weights omega _itrain by following formula:

Make learning rate $\mathrm{\&delta;}=\frac{1}{K},20\&le;K\&le;50;$

Weight training: $\left\{\begin{array}{cc}{\mathrm{\&omega;}}_i=(1-\mathrm{\&delta;}){\mathrm{\&omega;}}_i+\mathrm{\&delta;}& ,{\mathrm{\&Delta;}}_i<3\&times;{\mathrm{\&sigma;}}_i\\ {\mathrm{\&omega;}}_i=(1-\mathrm{\&delta;}){\mathrm{\&omega;}}_i& ,{\mathrm{\&Delta;}}_i\&GreaterEqual;3\&times;{\mathrm{\&sigma;}}_i\end{array}\right.,$

Average is trained: μ _i=(1-ρ) μ _i+ ρ X _t,

Standard deviation is trained: ${\mathrm{\&sigma;}}_i=\sqrt{(1-\mathrm{\&rho;}){\mathrm{\&sigma;}}_i^2+\mathrm{\&rho;}{(X_t-{\mathrm{\&mu;}}_i)}^2},$

Wherein, X _tbe the pixel color value of t two field picture, for intermediate variable;

(1c), after training terminates, its average mean value is calculated to each pixel then the M of an each pixel Gauss model is pressed weights omega _isort from big to small, replace the average of M Gauss model by the average of M-1 Gauss model, replace the average of M-1 Gauss model by the average of M-2 Gauss model, by that analogy, finally use replace the average of first Gauss model.

Step 2, to mate with the difference of Gaussian Background model frame according to the present frame of image shot by cell phone frame sequence and carries out sport foreground detection, obtain motion foreground picture.

With reference to Fig. 2, being implemented as follows of this step:

(2a) the color value v of each pixel of current frame image and the average μ of each Gauss model corresponding is with it calculated _iabsolute difference △ _i=| v-μ _i|, and by absolute difference and difference threshold T _c=3 × σ _icompare, if there is △ _i<T _c, then △ is upgraded _ithe average of corresponding Gauss model and standard deviation, on the contrary replace the average of the minimum Gauss model of weight with v, upgrade the weight of all Gauss models, concrete update mode is as follows:

Make intermediate variable α is the renewal coefficient in step (1a), ω _ibe the weight of i-th Gauss model,

Average upgrades: μ _i=(1-γ) μ _i+ γ X,

Standard deviation upgrades: ${\mathrm{\&sigma;}}_i=\sqrt{(1-\mathrm{\&gamma;}){\mathrm{\&sigma;}}_i^2+\mathrm{\&gamma;}{(X-{\mathrm{\&mu;}}_i)}^2},$

Weight upgrades: $\left\{\begin{array}{cc}{\mathrm{\&omega;}}_i=(1-\mathrm{\&alpha;}){\mathrm{\&omega;}}_i+\mathrm{\&alpha;}& ,{\mathrm{\&Delta;}}_i<T_c\\ {\mathrm{\&omega;}}_i=(1-\mathrm{\&alpha;}){\mathrm{\&omega;}}_i& ,{\mathrm{\&Delta;}}_i\&GreaterEqual;T_c\end{array}\right.,$

Wherein, X is the pixel color value of current frame image;

(2b) M of an each pixel Gauss model is sorted from big to small by weight, successively the weight of Gauss model is sued for peace from front to back, namely ω _jfor the weight of a jth Gauss model after sorting, wherein 0≤j≤B, when the value with value U is greater than weight and the threshold value T of setting _wtime, the weight number B of summation operation participated in record, 0.5<T _w<0.8, in a front B-1 Gauss model, if average μ _jdifference threshold T is less than with the absolute difference of color value v _c, namely | v-μ _j| <T _c, then the pixel of this present frame is judged to be background dot, otherwise is judged to be foreground point, complete sport foreground to detect, obtain motion foreground picture, as shown in Figure 6, wherein Fig. 6 (a) is current frame image, the motion foreground picture of Fig. 6 (b) for detecting.

Step 3, detects the motion foreground picture obtained from sport foreground and extracts moving target, generates and detects agglomerate list:

With reference to Fig. 3, being implemented as follows of this step:

(3a) target outermost layer contour detecting is carried out:

(3a1) from the upper left corner in sport foreground image, by from left to right, top-down order finds out its new frontier point, if can not find, complete all target outermost layer contour detecting in foreground image, otherwise using this new frontier point as initial boundary point S, completed the outermost layer contour detecting of the target comprising frontier point S by Contour extraction, make this frontier point S be current border point D;

(3a2) record the coordinate of current border point D, according to the boundary chain code direction of current border point D, find its adjacent boundary point N; With reference to Fig. 7, the specific implementation method finding the adjacent boundary point of current border point is as follows:

If current point is initial boundary point, be then that the direction of 4 is for initial direction is around being rotated counterclockwise the frontier point found in its eight neighborhood by fore boundary point with direction number, otherwise, a direction is rotated counterclockwise by finding the direction of current border point, be rotated counterclockwise around this current point the frontier point found in its eight neighborhood using this direction as initial direction, such as, if find the direction number of current point to be 5, then using direction number be the direction of 6 as initial direction, do around this current point the frontier point found counterclockwise in its eight neighborhood;

(3a3) judge whether to find adjacent boundary point N, if can not find, then complete the outermost layer contour detecting of the target comprising frontier point S, return step (3a1); Otherwise judge whether frontier point N overlaps with initial boundary point S, if overlapped; would complete the outermost layer contour detecting of the target comprising frontier point S, return step (3a1), otherwise using N as current border point D, return step (3a2);

(3b) characteristic information of the target outermost layer profile detected is calculated:

According to the point coordinate information of each target outermost layer profile, obtain the centre of form of this target these three features of wide W and high H:

$\stackrel{\&OverBar;}{i}=\frac{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{p}_n^{i}f(p_n^i,p_n^j)}{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}f(p_n^i,p_n^j)},$ $\stackrel{\&OverBar;}{j}=\frac{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{p}_n^{j}f(p_n^i,p_n^j)}{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}f(p_n^i,p_n^j)},$

$W=4\&times;\sqrt{\frac{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left(p_n^i\right)}^{2}f(p_n^i,p_n^j)}{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}f(p_n^i,p_n^j)}-{\left[\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{p}_n^{i}f(p_n^i,p_n^j)\right]}^2},$

$H=4\&times;\sqrt{\frac{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left(p_n^j\right)}^{2}f(p_n^i,p_n^j)}{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}f(p_n^i,p_n^j)}-{\left[\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{p}_n^{j}f(p_n^i,p_n^j)\right]}^2},$

Wherein, with be respectively x-axis and the y-axis coordinate of the n-th point, be the grey scale pixel value of the n-th point, N is the pixel number of target outermost layer profile;

(3c) according to the target outermost layer contour feature information obtained, according to closing on principle, polymerization process is carried out to target:

For any two targets that step (3a) obtains, as target A and target B, calculate its centre of form apart from Dis _aB:

${\mathrm{Dis}}_{\mathrm{AB}}=\sqrt{{({\stackrel{\&OverBar;}{i}}_A-{\stackrel{\&OverBar;}{i}}_B)}^2+{({\stackrel{\&OverBar;}{j}}_A-{\stackrel{\&OverBar;}{j}}_B)}^2},$

If then target A and target B polymerization is become target C, and delete target A and target B, the centre of form obtaining target C is wide W _c, high H _cbe respectively:

${\stackrel{\&OverBar;}{i}}_C=\frac{{\stackrel{\&OverBar;}{i}}_A+{\stackrel{\&OverBar;}{i}}_B}{2},$ ${\stackrel{\&OverBar;}{j}}_C=\frac{{\stackrel{\&OverBar;}{j}}_A+{\stackrel{\&OverBar;}{j}}_B}{2},$

$W_C=\frac{W_A+W_B}{2},$

$H_C=\frac{H_A+H_B}{2},$

Wherein, with be respectively x-axis and the y-axis coordinate of the target A centre of form, with be respectively x-axis and the y-axis coordinate of the target B centre of form, W _aand H _abe respectively the wide and high of target A, W _band H _bbe respectively the wide and high of target B, α _disfor the centre of form is apart from coefficient, its span is: 1.2≤α _dis≤ 1.4;

(3d) the outermost layer contour feature information of each target is described as lumpiness, and all lumpinesses that present frame obtains are arranged in order, generate and detect agglomerate list.

Step 4, utilizes feature matching method to carry out the tracking of motion agglomerate to the agglomerate detected in agglomerate list, upgrades and follow the tracks of agglomerate list.

With reference to Fig. 4, being implemented as follows of this step:

If it is empty for (4a) following the tracks of agglomerate list, then the detection agglomerate list of present frame is directly added to and follow the tracks of agglomerate list;

If (4b) following the tracks of agglomerate list is not sky, then travels through the detection agglomerate list of present frame and follow the tracks of agglomerate list, carrying out blob match by calculating the similarity detected in agglomerate list and the list of tracking agglomerate between each agglomerate;

If (4b1) detect an agglomerate A in agglomerate list to be greater than similarity threshold T with the similarity of the agglomerate B followed the tracks of in agglomerate list _s, then replace the characteristic information of agglomerate B with the characteristic information of agglomerate A, and agglomerate A is labeled as mates agglomerate, 0.4≤T _s≤ 0.6;

If the agglomerate A (4b2) detected in agglomerate list does not all mate with any one agglomerate followed the tracks of in agglomerate list, then agglomerate A is the agglomerate or the flase drop agglomerate that newly enter scene, is labeled as by this agglomerate A and does not mate agglomerate;

If (4b3) the agglomerate B followed the tracks of in agglomerate list can not be detected any one blob match in agglomerate list, then agglomerate B is for leaving scene agglomerate, is removed by this agglomerate B from the list of tracking agglomerate;

(4c) heavy decision threshold T is calculated _r:

$T_r=\frac{\frac{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\left|\mathrm{diff}(p_m^i,p_m^j)\right|}{M}+{\mathrm{\&sigma;}}_{(p_m^i,p_m^j)}\&times;{\mathrm{\&beta;}}_r}{2},$

Wherein with be respectively x and the y-axis coordinate of m point, for point the absolute value of the present frame at place and the grey scale pixel value difference of background frames, M is the number of having mated all pixels in agglomerate, for point locate the average of the standard deviation of each Gauss model, β _rattach most importance to decision threshold coefficient, its span is: 3≤β _r≤ 4.5;

(4d) heavy decision threshold T is utilized _r, using step (2b) to judge, that the method for pixel carries out major punishment to the foreground pixel point be labeled as in step (4b2) in the agglomerate do not mated is fixed, obtain major punishment surely than wherein, M _rthe pixel number of attaching most importance to still as foreground point after judging, M _afor not mating all pixel numbers in agglomerate;

(4e) utilizing the major punishment in step (4d) surely than judging this major punishment not mating agglomerate and whether should be added to and follow the tracks of in agglomerate list surely than correspondence, making T _sattach most importance to and judge than threshold value, if S _r≤ T _sthen remove this not mate and sentence agglomerate, otherwise this is not mated agglomerate and add to and follow the tracks of in agglomerate list, 0≤T _s≤ 0.4.

Step 5, utilizes and follows the tracks of the image information reparation that agglomerate list, sport foreground image and background model complete moving target.

With reference to Fig. 5, being implemented as follows of this step:

(5a) the target erasure rectangular area that user selectes is extracted, utilize the tracking agglomerate list of present frame that the pixel in this region is divided into two classes, one class is to follow the tracks of the motion pixel of locating in list, another kind of for not following the tracks of the static pixel of locating in list, described motion pixel belongs to the pixel needing to carry out target erasure;

(5b) use the motion pixel in the gray-scale value replacement shutter frame of relevant position pixel in background model frame, complete the image information reparation of moving target.

Effect of the present invention further illustrates by following emulation:

The texture-rich selected arbitrarily under outdoor elements, complex structure and under having the scene of apparent contour, use a smart mobile phone to fix a direction and manually take one section of frame of video, then by method of the present invention, moving target secondary in shutter frame is wiped and repaired, result as shown in Figure 8, wherein Fig. 8 (a) is for comprising the shutter frame image of secondary moving target, and Fig. 8 (b) is the result images repaired after erasing target.

As can be seen from Fig. 8 (b), the present invention can repair the image-region after shutter frame image motion target erasure well.

Claims (5)

1., based on the image shot by cell phone restorative procedure that motion detects, comprise the steps:

(1) use mixed Gaussian background modeling method, build the Gaussian Background model of mobile phone photographed scene, to mate with the difference of Gaussian Background model frame according to the present frame of image shot by cell phone frame sequence and carry out sport foreground detection, obtain motion foreground picture;

(2) detect the motion foreground picture obtained from sport foreground and extract moving target, generate and detect agglomerate list:

(2a) target outermost layer contour detecting is carried out:

(2a1) from the upper left corner in sport foreground image, by from left to right, top-down order finds out its new frontier point, if can not find, complete all target outermost layer contour detecting in foreground image, otherwise using this new frontier point as initial boundary point S, completed the outermost layer contour detecting of the target comprising frontier point S by Contour extraction, make this frontier point S be current border point D;

(2a2) record the coordinate of current border point D, according to the boundary chain code direction of current border point D, find its adjacent boundary point N ';

(2a3) judge whether to find adjacent boundary point N ', if can not find, then complete the outermost layer contour detecting of the target comprising frontier point S, return step (2a1); Otherwise judge whether frontier point N ' overlaps with initial boundary point S, if overlapped; would complete the outermost layer contour detecting of the target comprising frontier point S, return step (2a1), otherwise using N ' as current border point D, return step (2a2);

(2b) characteristic information of the target outermost layer profile detected is calculated:

According to the point coordinate information of each target outermost layer profile, obtain the centre of form of this target these three features of wide W and high H:

$\stackrel{\_}{i}=\frac{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{p}_n^{i}f(p_n^i,p_n^i)}{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}f(p_n^i,p_n^j)},$ $\stackrel{\_}{j}=\frac{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{p}_n^{j}f(p_n^i,p_n^i)}{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}f(p_n^i,p_n^j)},$

$W=4\&times;\sqrt{\frac{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left(p_n^i\right)}^{2}f(p_n^i,p_n^j)}{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}f(p_n^i,p_n^j)}-{\left[\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{p}_n^{i}f(p_n^i,p_n^j)\right]}^2},$

$H=4\&times;\sqrt{\frac{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{\left(p_n^j\right)}^{2}f(p_n^i,p_n^j)}{\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}f(p_n^i,p_n^j)}-{\left[\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{p}_n^{j}f(p_n^i,p_n^j)\right]}^2},$

Wherein, with be respectively x-axis and the y-axis coordinate of the n-th point, be the grey scale pixel value of the n-th point, N is the pixel number of target outermost layer profile;

(2c) according to closing on principle, polymerization process is carried out to target according to the target outermost layer contour feature information obtained:

For any two targets that step (2a) obtains, as target A and target B, calculate its centre of form apart from Dis _aB:

${\mathrm{Dis}}_{\mathrm{AB}}=\sqrt{{({\stackrel{\_}{i}}_A-{\stackrel{\_}{i}}_B)}^2+{({\stackrel{\_}{j}}_A-{\stackrel{\_}{j}}_B)}^2},$

If then target A and target B polymerization is become target C, and delete target A and target B, the centre of form obtaining target C is wide W _c, high H _cbe respectively:

${\stackrel{\_}{i}}_C=\frac{{\stackrel{\_}{i}}_A+{\stackrel{\_}{i}}_B}{2},{\stackrel{\_}{j}}_C=\frac{{\stackrel{\_}{j}}_A+{\stackrel{\_}{j}}_B}{2},$

$W_C=\frac{W_A+W_B}{2},$

$H_C=\frac{H_A+H_B}{2},$

Wherein, with be respectively x-axis and the y-axis coordinate of the target A centre of form, with be respectively x-axis and the y-axis coordinate of the target B centre of form, W _aand H _abe respectively the wide and high of target A, W _band H _bbe respectively the wide and high of target B, α _disfor the centre of form is apart from coefficient, its span is: 1.2≤α _dis≤ 1.4;

(2d) the outermost layer contour feature information of each target is described as lumpiness, and all lumpinesses that present frame obtains are arranged in order, generate and detect agglomerate list;

(3) utilize feature matching method to carry out the tracking of motion agglomerate to the agglomerate detected in agglomerate list, upgrade and follow the tracks of agglomerate list:

If it is empty for (3a) following the tracks of agglomerate list, then the detection agglomerate list of present frame is directly added to and follow the tracks of agglomerate list;

If (3b) following the tracks of agglomerate list is not sky, then travels through the detection agglomerate list of present frame and follow the tracks of agglomerate list, carrying out blob match by calculating the similarity detected in agglomerate list and the list of tracking agglomerate between each agglomerate;

(4) list of tracking agglomerate, sport foreground image and background model is utilized to complete the image information reparation of moving target.

2. image shot by cell phone restorative procedure as claimed in claim 1, use mixed Gaussian background modeling method, the Gaussian Background model of structure mobile phone photographed scene wherein described in step (1), carries out as follows:

(1a) M Gauss model is set up, by the average μ of first Gauss model to each pixel in video frame images ₀be initialized as the pixel value of the first two field picture, the average μ of all the other Gauss models _ibe initialized as random value, 0 < i < M; All Gauss model standard deviation sigma are initialized as 10, by the weights omega of first Gauss model ₀be set to 0.001 < α < 0.01, the weights omega of all the other Gauss models _ibe set to 0 < i < M;

(1b) in K frame to the average μ of each Gauss model _i, standard deviation sigma _iand weights omega _itrain, 20≤K≤50, at the end of training, its average mean value is calculated to each pixel then the M of an each pixel Gauss model is pressed weights omega _isort from big to small, replace the average of M Gauss model by the average of M-1 Gauss model, replace the average of M-1 Gauss model by the average of M-2 Gauss model, by that analogy, finally use replace the average of first Gauss model.

3. image shot by cell phone restorative procedure as claimed in claim 1, the present frame according to image shot by cell phone frame sequence wherein described in step (1) mates with the difference of Gaussian Background model frame and carries out sport foreground detection, obtain motion foreground picture, carry out as follows:

(1c) each Gauss model average μ that the color value v of each pixel of current frame image is corresponding is with it calculated _iabsolute difference Δ _i=| v-μ _i|, and by absolute difference and difference threshold T _c=3 × σ _icompare, if there is Δ _i< T _c, then Δ is upgraded _ithe average of corresponding Gauss model and standard deviation, on the contrary the average of replacing the minimum Gauss model of weight with v, upgrades the weight of all Gauss models;

(1d) M of an each pixel Gauss model is sorted from big to small by weight, successively the weight of Gauss model is sued for peace from front to back, namely ω _jfor the weight of a jth Gauss model after sorting, wherein 0≤j < B, when the value with value U is greater than weight and the threshold value T of setting _wtime, the weight number B of summation operation participated in record, 0.5 < T _w< 0.8, in a front B-1 Gauss model, if average μ _jdifference threshold T is less than with the absolute difference of color value v _c, namely | v-μ _j| < T _c, then the pixel of this present frame is judged to be background dot, otherwise is judged to be foreground point, complete sport foreground and detect, obtain motion foreground picture.

4. image shot by cell phone restorative procedure as claimed in claim 1, carries out blob match by calculating the similarity detected in agglomerate list and the list of tracking agglomerate between each agglomerate wherein described in step (3b), carries out as follows:

If (3b1) detect an agglomerate A in agglomerate list to be greater than similarity threshold T with the similarity of the agglomerate B followed the tracks of in agglomerate list _s, then replace the characteristic information of agglomerate B with the characteristic information of agglomerate A, and agglomerate A is labeled as mates agglomerate, 0.4≤T _s≤ 0.6;

If the agglomerate A (3b2) detected in agglomerate list does not all mate with any one agglomerate followed the tracks of in agglomerate list, then agglomerate A is the agglomerate or the flase drop agglomerate that newly enter scene, is labeled as by this agglomerate A and does not mate agglomerate;

If (3b3) the agglomerate B followed the tracks of in agglomerate list can not be detected any one blob match in agglomerate list, then agglomerate B is for leaving scene agglomerate, is removed by this agglomerate B from the list of tracking agglomerate;

(3b4) heavy decision threshold T is calculated _r:

$T_r=\frac{\frac{\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\left|\mathrm{diff}(p_m^i,p_m^j)\right|}{M}+{\mathrm{\&sigma;}}_{(p_m^i,p_m^j)}\&times;{\mathrm{\&beta;}}_r}{2},$

Wherein with be x-axis and the y-axis coordinate of m point, for point the absolute value of the present frame at place and the grey scale pixel value difference of background frames, M is the number of having mated all pixels in agglomerate, for point locate the average of the standard deviation of each Gauss model, β _rattach most importance to decision threshold coefficient, its span is: 3≤β _r≤ 4.5;

(3b5) heavy decision threshold T is utilized _r, using step (1d) to judge, that the method for pixel carries out major punishment to the foreground pixel point be labeled as in step (3b2) in the agglomerate do not mated is fixed, obtain major punishment surely than wherein, M _rthe pixel number of attaching most importance to still as foreground point after judging, M _afor not mating all pixel numbers in agglomerate;

(3b6) utilizing the major punishment in step (3b5) surely than judging this major punishment not mating agglomerate and whether should be added to and follow the tracks of in agglomerate list surely than correspondence, making T _sattach most importance to and judge than threshold value, if S _r≤ T _sthen remove this not mate and sentence agglomerate, otherwise this is not mated agglomerate and add to and follow the tracks of in agglomerate list, 0≤T _s≤ 0.4.

5. image shot by cell phone restorative procedure as claimed in claim 1, the image information reparation that agglomerate list is followed the tracks of in the utilization wherein described in step (4), sport foreground image, background model complete moving target, carry out as follows:

(4a) the target erasure rectangular area that user selectes is extracted, utilize the tracking agglomerate list of present frame that the pixel in this region is divided into two classes, one class is to follow the tracks of the motion pixel of locating in list, another kind of for not following the tracks of the static pixel of locating in list, described motion pixel belongs to the pixel needing to carry out target erasure;

(4b) use the motion pixel in the gray-scale value replacement shutter frame of relevant position pixel in background model frame, complete the image information reparation of moving target.