CN102831609B - Graphcut-based switch mode image matting technology - Google Patents

Abstract

The invention discloses a graphcut-based switch mode image matting method. The graphcut-based switch mode image matting method includes the steps: (1) inputting an initial image to compute a covariance matrix, diagonalizing the computed covariance matrix to obtain a diagonalization matrix, judging whether the percentage of nonzero elements on diagonalization matrix diagonals accounting for the total elements exceeds a predetermined threshold or not, executing the step (2) if yes, and executing the step (3) by skipping the step (2) if no; (2), performing dimensionality reduction for the image by the aid of principal component analysis; (3) computing the probability density gradient of the image subjected to dimensionality reduction or the initial image not subjected to dimensionality reduction due to the fact the judgment result is yes by the aid of a camshift method, and clustering; (4) regarding the image processed in the step (3) as a graph, and marking energy for all classes in the graph; (5) performing graph cut according to the energy minimization principle; and (6) repeating the steps (3), (4) and (5) until the predetermined condition of convergence is satisfied to obtain matting results. The method has the advantages of high operating efficiency and accurate matting.

Description

A kind of switching regulator based on Graphcut is scratched diagram technology

Technical field

The invention belongs to Computer Image Processing field, particularly relate to a kind of switching regulator based on Graphcut and scratch diagram technology.

Background technology

Digital matting is a kind of technology that the prospect of image part is separated from background, it passes through a small amount of part prospect and background area in user's specify image, and according to certain decision rule, isolates automatically, exactly all foreground objects according to these promptings.Scratching figure is requisite gordian technique in production of film and TV with image synthetic technology, is widely used in media production.Digital matting can be divided into again blue screen according to the difference of original image and scratch the stingy figure of figure, natural image matting, shadow matting and environment etc., and what this patent was mainly studied is background natural image matting arbitrarily.

Natural image matting develops into and has produced many different algorithms today.Rotoscoping is a kind of natural image matting technology of generally using early proposing, but this technology too relies on operating personnel's experience, and workload is large, and stingy figure effect is good not.The stingy drawing method of Autokey has improved rotoscoping process, in the method, can obtain the boundary curve of foreground object in each frame by rotoscoping, proofreaies and correct afterwards the boundary curve of each frame by a small amount of manual work.AutoKey has adopted a kind of adaptive emergence scheme, is relatively applicable to the situation of the edge comparison " firmly " of foreground object, and the many complex edges of inapplicable burr.In stingy figure field, there are many natural image matting methods recently, mainly contained Knockout method, Ruzon-tomasi method, Bayesian method, Poisson method and Grabcut method.

Natural image matting can be divided into region division, color is estimated estimates 3 steps with α, first carries out trimap division, then asks foreground composition and the α value of each point in zone of ignorance.Knockout method is simple, fast operation, but is only applicable to the stingy figure of smooth image; Ruzon-tomasi method adopts statistical method to carry out estimated value, but the calculated amount that the color of the method is estimated and α estimates is very large, and its processing speed is very slow; Chuang has proposed that a stingy drawing method based on Bayesian frame---Bayesian scratches drawing method, although this method speed, effect is unsatisfactory.Sun etc. have proposed Poisson and have scratched drawing method, first coloured image is converted into gray level image and obtains trimap, again using two outline lines as initial boundary condition, in the gradient fields of original image, set up a Poisson equation, be applicable to the stingy figure of the image of change color smoother, when image more complicated, cover the gradient fields of picture and the gradient fields of original image differs greatly, Poisson is scratched figure can not solve this situation; In Grabcut, user is along rectangle of surrounding's picture of foreground object, then by image, cut apart the mode that adds eclosion process and pluck out exactly foreground object, while making to scratch figure in this way, the necessary smoother of background color in rectangle, and near color edge differ must be larger, the edge of foreground object can not be too complicated.

Summary of the invention

Technical matters to be solved by this invention is, operational efficiency is high, scratch and scheme switching regulator accurately and scratch drawing method.

The present invention solves the problems of the technologies described above by following technological means:

Switching regulator based on Graphcut is scratched a drawing method, comprises the following steps:

1) input original image, calculate covariance matrix, the covariance matrix calculating is carried out to diagonalization and obtain diagonalizable matrix, judge whether the number percent that nonzero element on diagonalizable matrix diagonal line accounts for total element surpasses predetermined threshold value, if, perform step 2), if not, skips steps 2) and execution step 3);

2) utilize principal component analysis (PCA) (PCA) to carry out dimension-reduction treatment to image;

3) to the image after dimension-reduction treatment or because of step 1) judgment result is that it is the original image without dimension-reduction treatment, use camshift method calculating probability density gradient and carry out cluster;

4) by through step 3) image after processing regards a figure as, to class mark energy all in figure;

5) according to energy minimization principle, carrying out figure cuts;

6) repeating step 3), 4) and 5) until meet the predetermined condition of convergence, obtain scratching figure result.

Preferably:

Described predetermined threshold value is 50%.

Described step 3) comprise the following steps:

(3.1) in image, choose search window;

(3.2) calculate the barycenter of zeroth order square, search window;

(3.3) adjust search window size;

(3.4) mobile search Chuan center is to barycenter, if displacement is greater than default fixed threshold, repeat 3.2), 3.3) and 3.4), until the displacement between search Chuan center and barycenter is less than default fixed threshold, or the number of times of loop computation reaches a certain maximal value, stop calculating.Wherein, described fixing threshold value can select 5 pixels also can suitably adjust according to actual needs, and the number of times of described loop computation is not more than 4, and preferably 3 or 4.

Described step 1) comprising: input original image, regard the image of input as data set with matrix representation, deduct Mean Matrix, making the average of data set in each dimension is zero, calculate covariance matrix, the covariance matrix calculating is carried out to diagonalization, obtain diagonalizable matrix, judge whether the number percent that nonzero element on diagonalizable matrix diagonal line accounts for total element surpasses predetermined threshold value, if not, perform step 2), if so, skips steps 2) and execution step 3); Described step 2) comprising: described covariance matrix is carried out to feature decomposition, calculate its major component proper vector, the major component proper vector calculating is taken advantage of in raw data set vector, obtain the data set after projection, then add the above Mean Matrix, obtain the data after dimensionality reduction.Wherein, the object that deducts Mean Matrix is, while making subsequent calculations covariance matrix, can obtain the sparse matrix that a major part is neutral element, is convenient to process.

The described predetermined condition of convergence refers to that energy is without significantly decay.Preferably energy function reduced to be controlled in the scope that is no more than 5%.

Compared with prior art, beneficial effect of the present invention comprises:

1, utilize diagonal angle nonzero element in covariance matrix to judge in image, whether most information concentrates in less several major components, selects adaptively preprocess method, realize dissimilar Images Classification is processed, make the dimensionality reduction effect optimization of PCA.

2, use PCA method to extract main proper vector in figure, select less major component to carry out information in presentation graphs, not only can be used as Feature Dimension Reduction pre-service, cluster segmentation operand after making it reduces at double, simultaneously, because the major component being arranged in eigenvalue spectrum below has often reflected the noise in data, so the pre-service that PCA the carries out noise effect in removal of images to a certain extent.

3, utilize camshift algorithm to image pixel cluster, automatically adjust search window, utilize the characteristic of the self-adaption gradient rising search peak of camshift to try to achieve the probability density gradient of image, by image divide into several classes, realize and effectively from complicated background area, extract more exactly target object.

4, iteration carries out pixel cluster mark and minimizes figure cutting, until ENERGY E (X) is thought when significantly decaying, calculates convergence, automatically stops iteration, guarantees that obtaining E (X) converges to minimum value, obtains optimum stingy figure result.

Accompanying drawing explanation

Fig. 1 is the process flow diagram of the specific embodiment of the invention.

Embodiment

Preferred embodiment the invention will be further described to contrast accompanying drawing combination below.

Input original image, regards the image of input as data set with matrix representation, deducts average, and making the average of data set in each dimension is zero.Calculate covariance matrix:

$C_x=\frac{1}{n-1}{\mathrm{XX}}^T$

C _xbe square symmetric matrix of a m * m, m is the proper vector number of observational variable, and the element on diagonal line is the variance of corresponding observational variable, and the element on off-diagonal is the covariance between corresponding observational variable.Element on diagonal line is larger, shows that signal is stronger, and the importance of variable is higher; Element on diagonal line less showing may be noise or the secondary variable existing.Element size on off-diagonal corresponding to dependent observation variable between the size of redundancy degree.

If prospect and background area are distributed as the strip that is similar to cigar shape in RGB color space, this shape is mainly distributed near diagonal line the nonzero element of the covariance matrix of sample, is applicable to principal component analysis (PCA).The covariance matrix calculating is carried out to diagonalization, obtain diagonalizable matrix C _y, judgement C _ywhether the nonzero element on diagonal line is no more than half of total element, if be no more than, the information of key diagram picture concentrates on front several major component element, can utilize PCA method to carry out dimension-reduction treatment, if surpassed, illustrate that the information distribution of this image comparatively disperses, use the PCA method meeting of processing loss important information, therefore skip the 2nd step, directly carry out subsequent treatment.

2, use PCA method to carry out dimension-reduction treatment to image.Pixel forms the intermediate color between identical basic look or two kinds of colors under different brightness conditions, use PCA method to find the main shaft through each bunch of center, " bunch " refer to the strip region that prospect or background distribute and are polymerized at rgb space, utilize the set of PCA conversion calculate along main shaft bunch scope, find out along the border (r of the set of main shaft conversion _min, r _max), calculate the average [μ in every one dimension PCA space ₁, μ ₂, μ ₃], by the terminal P of conversion _min=[r _min, μ ₂, μ ₃] and P _max=[r _max, μ ₂, μ ₃] carry out inversion and change to and in rgb space, form P ₀and P ₁.

Suppose uncertain be that each pixel s in the set U of pixel of prospect or background is comprised of a kind of background colour that approaches color b, foreground approaches color f.B is vector on a bit, f be vector on a bit, b and f are the estimated colors of background and prospect.Selecting b and the optimal point of f is the point that approaches s most, calculates the q of prospect with following formula _fq with background _d:

$q=\frac{(s-P_0)\·(P_1-P_0)}{{|P_1-P_0|}^2}$

If q _fand q _bin scope (0,1), so

b＝P _1bq _b+P _0b(1-q _b)

f＝P _1fq _f+P _0f(1-q _f)

Make f point and b point constrain in P ₀and P ₁between.

α value is calculated with following formula:

$\mathrm{\α}=\frac{(s-b)\·(f-b)}{{|f-b|}^2}$

α value is constrained in (0,1).

Calculate above-mentioned vector value, covariance matrix is carried out to feature decomposition, ask for proper vector and corresponding characteristic root, afterwards, in rgb space, travel through, find the vector of an eigenwert maximum, P is made in order ₁.At P ₁travel through with vertical vector space, find out time large vector corresponding to eigenwert, be denoted as P ₂.To above process circulation, until find out the vector of whole m.The namely sequence of " pivot " of order that they generate, according to eigenmatrix, obtain new data set like this, this patent fetches data and concentrates 80% major component, and the major component proper vector premultiplication calculating, with raw data set vector, is obtained to the data set after projection.Due in first calculated, data have been carried out deducting equal Value Operations, therefore, need to the result on calculate after, add Mean Matrix, recover raw data.

3, through the d dimension space R after dimensionality reduction pre-service ^dmiddle n sample point x _i, i=1 ..., n, the camshift vector of ordering at x is:

$C_h\left(x\right)=\frac{1}{k}\underset{x_i\∈S_h}{\mathrm{\Σ}}(x_i-x)---\left(1\right)$

Wherein, S _hbe the higher-dimension ball region that a radius is h, meet the set that the y of following relation is ordered,

S _h(x)≡{y:(y-x) ^T(y-x)≤h ²} (2)

K is illustrated in this n sample point x _iin, there is k point to fall into S _hin region.(x _i-x) be sample point x _iwith respect to the offset vector of an x, the camshift vector C of (1) formula definition _h(x) be exactly to falling into region S _hin k sample point with respect to the offset vector summation of an x and then average.From intuitively, if sample point x _ifrom a probability density function f (x), sample and obtain, because the probability density gradient of non-zero is pointed to probability density and is increased maximum direction, from average, S _hsample point in region more drops on along the direction of probability density gradient.Therefore, corresponding camshift vector C _h(x) point to the direction of probability density gradient:

$C_h\left(x\right)=\frac{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}G\left(\frac{x_i-x}{h}\right)w\left(x_i\right)x_i}{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}G\left(\frac{x_i-x}{h}\right)w\left(x_i\right)}-x---\left(3\right)$

Wherein: G (x) Shi Yige unit kernel function, w (x _i)>=0 is one and is assigned to sampled point x _iweight

First of above formula the right, be designated as c _h(x),

$c_h\left(x\right)=\frac{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}G\left(\frac{x_i-x}{h}\right)w\left(x_i\right)x_i}{\underset{i=1}{\overset{n}{\mathrm{\Σ}}}G\left(\frac{x_i-x}{h}\right)w\left(x_i\right)}$

The execution of camshift algorithm circulation is step below:

1) a given initial point x (barycenter of search window), search window initial size h, kernel function G (X), allowable error ε,

2) calculate c _h(x), zeroth order square $M_{00}=\underset{x}{\mathrm{\Σ}}\underset{y}{\mathrm{\Σ}}I(x,y)$

3) according to zeroth order square adjustment search window size, mobile barycenter, c _h(x) be assigned to x

4) if || c _h(x)-x|| < ε, end loop; If not, continue to carry out 2)

By (3) formula, known c _h(x)=x+C _h(x), therefore the step above namely constantly moves along the gradient direction of probability density, step-length is not only relevant with the size of gradient simultaneously, also relevant with the probability density of this point, in the large place of density, the peak value of the more approaching probability density that will look for, camshift algorithm makes mobile step-length smaller, on the contrary, in the little place of density, mobile step-length is just larger. meeting under certain condition, camshift algorithm is bound to converge near the peak value of this point, the starting point that converges to same point is classified as to a class, then the label of this class is assigned to these starting points, just obtain image and be divided into the result after some classes.

4, regarding a figure, G={V, ε as through the image that step process is crossed before }, V is all classes, and ε is the limit that connects adjacent class, and the class in figure and limit are done to two meta-tag, and each i ∈ V, has a unique x _i{ prospect is 1 to ∈, and background is that 0} is corresponding with it.According to user's original input, the class set in figure is divided into prospect class set F and background class set B, unknown class set U.To class x all in figure _imark ENERGY E (X):

$E\left(X\right)=\underset{i\&Element;V}{\mathrm{\&Sigma;}}{E}_1\left(x_i\right)+\mathrm{\&lambda;}\underset{(i,j)\&Element;E}{\mathrm{\&Sigma;}}{E}_2(x_i,x_j)$

Calculate each node i to the minor increment of each prospect class with corresponding back pitch from $d_i^B=\min \left|\right|C\left(i\right)-B\left|\right|.$

If judgement belongs to the prospect class of user's appointment, mark E ₁(x _i=F)=0, E ₁(x _i=B)=∞, if belong to background classes, mark E ₁(x _i=F)=∞, E ₁(x _i=B) if=0 belong to zone of ignorance, use and determining the mark of unknown class with color phase recency front, background.In addition, one of the definition function E with gradient correlation method ₂, reduce the variation of mark between class that color is close, its variation is only occurred on border.

Defined formula:

$\left\{\begin{array}{c}{E}_1(x_i=1)=0E_1(x_i=0)=\&infin;\&ForAll;i\&Element;F\\ E_1(x_i=1)=\&infin;E_1(x_i=0)=0\&ForAll;i\&Element;B\\ E_1(x_i=1)=\frac{d_i^F}{d_i^F+d_i^B}{E}_1(x_i=0)=\frac{d_i^B}{d_i^F+d_i^B}\&ForAll;\&Element;U\\ E_2(x_i,x_j)=e^{-\mathrm{\&beta;}\left(\right||c_i-c_j|\left|\right)}\end{array}\right.$

5, the edge e ∈ ε of each class has been assigned with a weight w in the drawings _e, once to cut be exactly to obtain two terminal nodes in figure to separate the subset C of required edge aggregation of cutting off to figure, defines once the cost that figure cuts to be:

$\left|C\right|=\underset{e\&Element;C}{\mathrm{\&Sigma;}}{w}_e$

Node in figure has represented class, and edge has represented the relation of closing between class, and it is to make cost function minimum that optimum figure cuts result.

6, iteration is carried out 3,4,5 steps, and calculating probability density gradient, cluster mark and cut apart, until E (X) iteration when significantly decaying stops, assert that convergence obtains satisfied stingy figure result.Above content is in conjunction with concrete preferred implementation further description made for the present invention, can not assert that specific embodiment of the invention is confined to these explanations.For those skilled in the art, without departing from the inventive concept of the premise, can also make some being equal to substitute or obvious modification, and performance or purposes identical, all should be considered as belonging to protection scope of the present invention.

Claims (3)

1. the switching regulator based on Graphcut is scratched a drawing method, it is characterized in that, comprises the following steps:

1) input original image, regard the image of input as data set with matrix representation, deduct Mean Matrix, making the average of data set in each dimension is zero, calculate covariance matrix, the covariance matrix calculating is carried out to diagonalization, obtain diagonalizable matrix, judge whether the number percent that nonzero element on diagonalizable matrix diagonal line accounts for total element surpasses 50%, if not, perform step 2), if so, skips steps 2) and execution step 3);

2) described covariance matrix is carried out to feature decomposition, calculate its major component proper vector, the major component proper vector calculating is taken advantage of in raw data set vector, obtain the data set after projection, then add the above Mean Matrix, obtain the image after dimension-reduction treatment, described raw data set vector refers to the vector that represents original image;

3) to the image after dimension-reduction treatment or because of step 1) judgment result is that it is the original image without dimension-reduction treatment, use camshift method calculating probability density gradient and carry out cluster;

4) by through step 3) image after processing regards a figure as, to class mark energy all in figure;

5) according to cost function minimization principle, carrying out figure cuts;

6) repeating step 3), 4) and 5) until meet the predetermined condition of convergence, obtain scratching figure result;

Wherein, described step 3) comprise the following steps:

3.1) in image, choose search window;

3.2) calculate zeroth order square, the barycenter of search window;

3.3) adjust search window size;

3.4) mobile search Chuan center is to barycenter, if displacement is more than or equal to default fixed threshold, repeat 3.2), 3.3) and 3.4), until the displacement between search Chuan center and barycenter is less than default fixed threshold, or the number of times of loop computation reaches predetermined maximal value, stop calculating.

2. the switching regulator based on Graphcut according to claim 1 is scratched drawing method, it is characterized in that: described predetermined maximal value is 3 or 4.

3. the switching regulator based on Graphcut according to claim 1 is scratched drawing method, it is characterized in that: what the described predetermined condition of convergence was energy function reduces to be no more than 5%.