CN103247050B - A kind of progressive picture dividing method - Google Patents

Abstract

The invention belongs to computer vision, computer graphics and image procossing crossing domain, disclose a kind of progressive picture dividing method of Shape-based interpolation priori.The present invention is an iteration, gradual method, and user often adds a raw curve, will trigger the optimizing process of a Shape-based interpolation priori, and every suboptimization all can Real-time Feedback segmentation result.The method of each iteration comprises: sketch the contours raw curve, matched curve, adds surplus capacity, definition energy function, definition data item, and definition shape prior item, definition gradient terms, solves energy function.The progressive picture dividing method of Shape-based interpolation priori that the present invention proposes, often step operation can have real-time result feedback, can ensure to obtain the segmentation result that user wants.In addition, the method for the invention can segmentation object thing from image easily, even if front, background color is seriously overlapping, when obscure boundary is clear, also can obtain satisfied segmentation result.

Description

A kind of progressive picture dividing method

Technical field

The invention belongs to computer vision, computer graphics and image procossing crossing domain, relate to a kind of progressive picture dividing method of Shape-based interpolation priori.A given image, user provides shape constraining information by designed user interface, in conjunction with the gradient information of shape constraining and image itself, adopts figure segmentation method computed segmentation result.

Background technology

Iamge Segmentation is the process split from image by interested object.Iamge Segmentation has important using value for the research field taking object as fundamental element, and such as, two-dimensional image changed into three-dimension, makes animation from two dimensional image, and medical image analysis and modeling etc.In the optimization algorithm of image segmentation problem, figure cuts algorithm (graphcut) and is used widely owing to calculating globally optimal solution.Figure cuts algorithm and will give the mark that in image, each pixel one is unique, i.e. prospect or background, thus realizes segmentation.Employing figure cuts the key that algorithm carries out Iamge Segmentation how to set up objective function, and the defining relation of objective function is to the quality of segmentation effect.

Current, cut most image partition methods of algorithm based on figure, be by before user interactions specified portions, background, then according to the partial analysis of specifying and before setting up, the color model of background.For the pixel of unknown portions, in color space, weigh it as prospect or the possibility of background with distance that is front, background model according to it.These class methods can obtain good result in sharply marginated situation.But the most of image in reality does not meet such condition.Tempering of some ancient painting elapsed time, the serious and obscurity boundary of picture noise.Even if in the photo of camera shooting, due to different target thing usually color similarity, also segment boundary fuzzy can be caused.From this kind of image, how to extract object of interest thing easily become a problem demanding prompt solution.

Summary of the invention

What exist for current image partition method is difficult to the problems such as partitioning boundary blurred picture, the present invention proposes a kind of progressive picture dividing method of Shape-based interpolation priori, can segmentation object thing from image easily, also can obtain satisfied segmentation result when processing in-defined boundary.

Technical scheme of the present invention is: adopt alternately simple, freeform lines as input form, user by touch-screen, Digitizing plate even mouse sketch the contours along object border.The line that every bar newly adds is called raw curve, namely without the curve of any process.Be a series of continuous print nodes by curve record.Then, by it under consideration noise situations, shaping pattern curve part is fused to.If the raw curve of Article 1, then direct as pattern curve.Pattern curve is also made up of a series of node, and each node is recording curve width herein simultaneously, namely forms the number of the raw curve of this node.Then, updated pattern curve form is turned to shape constraining item, using the weight of node width as the shape prior of Nodes, combining image gradient information, structure energy function.Finally, employing figure cuts Algorithm for Solving function minimum, obtains segmentation result.User often adds a raw curve all can perform above-mentioned steps, and user, by observing the result of Real-time Feedback, adds new raw curve to upgrade pattern curve, to realize segmentation result optimization.

Compared with prior art, innovation of the present invention is: (1) proposes the gradual dividing method of Shape-based interpolation priori, and often step operation can have real-time result feedback, can ensure to obtain the segmentation result that user wants; (2) propose in the form lines freely and, as input, formed the convenient strategy of shape prior by matching; (3) energy function constructed, at different time, locus place, for the weight of adjustable shape priori and gradient constraint, can be controlled by user and change.The present invention can tackle the different situation of diverse location sharpness of border degree, simultaneously can by adding raw curve, changes partial weight and excites a new iterative computation, Optimized Segmentation result.

Accompanying drawing explanation

Fig. 1 is the process flow diagram of the method for the invention;

Fig. 2 is the experimental result of application example of the present invention: (a) is the image of input and raw curve, and (b) is segmentation result.

Embodiment

Below in conjunction with the drawings and specific embodiments, the present invention will be further described.

The present invention is an iteration, gradual method, and user often adds a raw curve, will trigger the optimizing process of a Shape-based interpolation priori, and every suboptimization all can Real-time Feedback segmentation result.The method flow diagram of each iteration as shown in Figure 1, specifically comprises the following steps:

Step one, upgrades pattern curve.

The present invention adopts the mode of the raw curve of matching to form pattern curve.Easy more than sketching the contours long curve in view of sketching the contours funiclar curve, the present invention advises that funiclar curve inputs as user, but does not limit the length of input curve.Merging the another one advantage generating pattern curve is can adjust pattern curve by adding new curve in localized positions.Specifically comprise following content:

(1) raw curve is sketched the contours

Read in image in the user interface, user sketches the contours raw curve with touch-screen, Digitizing plate or mouse along the edge of interested object.All raw curves must be consistent with the start-stop order of pattern curve, such as, all follows counterclockwise or clock-wise order.This order be in step 2 before, the foundation of the orientation of background supposition.

(2) matched curve

If current iteration belongs to the first round of fighting, then raw curve inherently pattern curve; If non-first leg, namely there is partial shape curve, then upgraded pattern curve by the mode of the raw curve of matching and existing pattern curve.

Two class curves form by a series of node.Represent new hyperplasia curve with S, use represent a front iteration, i.e. the pattern curve that formed of t-1 moment, n=1,2 ..., N, N are the number of node; represent that the n-th node is engraved in the position on image when t-1; represent the width of this Nodes curve, namely form the number of this node raw curve used; Assuming that C ^t-1the n-th node corresponding point on S are expressed as q _n, q _ni.e. C ^t-1? the normal at place and the intersection point of curve S; Then current iteration, i.e. the pattern curve C of t ^tthe position of the n-th node on image with the curve width of this Nodes be respectively:

$p_n^t=0.5p_n^{t-1}+0.5q_n$

$w_n^t=w_n^{t-1}+1$

Consider that the raw curve of Freely input is often comparatively mixed and disorderly, regulation: if q _nwith distance be greater than certain threshold value (suggestion is a 0.5 δ pixel, and δ is the constant in units of pixel), corresponding point not as merging point consider, then:

$p_n^t=p_n^{t-1}$

$w_n^t=w_n^{t-1}$

(3) surplus capacity is added

If S and C ^t-1possess lamination portion and have surplus capacity, then adding node corresponding for surplus capacity to C ^tin, and at C ^tin local smooth treatment is carried out to the junction of the part that the part of adding as surplus capacity and matching generate.

For the border of object to be split clearly, pattern curve need not be fitted border completely, and gradient information can play conclusive effect in the determination of partitioning boundary; For unsharp border, user needs the mode by adding raw curve more, more adequately designated shape curve, and improves the weight of shape prior.

Step 2, constructs and solves energy function.

By in current iteration process, renewal and as surplus capacity add partial shape curve be expressed as C'.Only around C', perform segmentation, before the pixel that the C' that namely adjusts the distance is less than δ is carried out, background class.The value of δ is related to computation complexity, is also related to the search area of real partitioning boundary simultaneously, and in order to balance the two, suggestion δ gets 20 pixels.Specifically comprise following content:

(1) energy function is defined

Involved pixel is expressed as figure G=< ν, ε >, wherein ν represents all pixel set, and ε represents the limit between neighbor.Target is that these pixels are divided into prospect and background, to find the real border through these pixels.Namely x is used _imark each pixel.X _i{ 1,0} represents prospect and background to ∈ respectively.Definition energy function is as follows:

$E\left(X\right)=\underset{i\&Element;\mathrm{\&nu;}}{\&Sigma;}{E}_1\left(x_i\right)+\underset{(i,j)\&Element;\mathrm{\&epsiv;}}{\&Sigma;}\&lsqb;{\mathrm{\&omega;}}_{i-j}{E}_2(x_i,x_j)+E_3(x_i,x_j)\&rsqb;$

In formula, E ₁(x _i) be data item, expression is by front, background x _i{ 1,0} marks the cost in each pixel to ∈; E ₂(x _i, x _j), E ₃(x _i, x _j) represent cost when different marks being assigned to adjacent node, wherein, E ₂(x _i, x _j) be shape prior item, E ₃(x _i, x _j) be gradient terms.

(2) data item is defined

The node of graph of the left and right side (according to the starting and terminal point position judgment left and right sides of C') of C' in figure G is expressed as L and R, is expressed as from C' one deck pixel farthest with preset along " right side-prospect, left side-background " on pattern curve start-stop direction, or the supposition of " left side-prospect, right side-background ".The follow-up explanation of the present invention will with " right side-prospect, left side-background " for supposition.Setting belongs to pixel be absolute foreground pixel, belong to pixel be absolute background pixel.Definitely, background constraint is in order to avoid figure cuts the minimal cut phenomenon of algorithm.These pixels will not change state after segmentation optimizing process.Segmentation will be right other pixels do before, context marker.E ₁(x _i) be defined as follows:

$\left\{\begin{array}{ccc}{E}_1(x_i=1)=0& E_1(x_i=0)=\mathrm{\&infin;}& i\&Element;{\mathrm{\&Omega;}}_R^{C^{\&prime;}}\\ E_1(x_i=1)=\mathrm{\&infin;}& E_1(x_i=0)=0& i\&Element;{\mathrm{\&Omega;}}_L^{C^{\&prime;}}\\ E_1(x_i=1)=0& E_1(x_i=0)=0& i\&Element;\left\{\mathrm{\&nu;}-{\mathrm{\&Omega;}}_R^{C^{\&prime;}}-{\mathrm{\&Omega;}}_L^{C^{\&prime;}}\right\}\end{array}\right.$

(3) shape prior item is defined

E ₂(x _i, x _j) be defined as:

$E_2(x_i,x_j)=\frac{d_{i-j}^{C^{\&prime;}}}{\mathrm{\&delta;}}\left|x_i-x_j\right|$

In formula, represent the distance of the sub-pixel between pixel i and j to C', in energy function, be positioned at E ₂(x _i, x _j) front side ω _i-jthe weight of dynamic adjustments shape prior in cutting procedure, the width of the node that the normal distance of sub-pixel on C' between up-to-date moment pixel i and the j being defined as iteration is nearest.Normal distance on image on certain point and C' between certain node is defined as the distance of this point to this node normal.| x _i-x _j| represent shape prior only (i.e. x on statistical boundary _i≠ x _j) value.

(4) gradient terms is defined

E ₃(x _i, x _j) be defined as:

$E_3(x_i,x_j)=\frac{\mathrm{\&lambda;}}{1+D_{i,j}}\left|x_i-x_j\right|$

In formula, D _i,j=(r _i-r _j) ²+ (g _i-g _j) ²+ (b _i-b _j) ²represent pixel i, the j distance in rgb space, r _i, g _i, b _irepresent the RGB component of pixel i respectively.λ is a constant, for equilibrium gradient item and initial shape prior item.Gradient terms is (i.e. x on statistical boundary equally only _i≠ x _j) value.

(5) energy function is solved

The present invention adopts figure to cut the minimum value of Algorithm for Solving energy function, obtains local segmentation result.After pattern curve is closed, as sketched the contours and the pattern curve formed according to counter clockwise direction under the supposition of " right side-prospect, left side-background ", after singulation, the part that divided border comprises will be prospect, and other parts divide background into.

Provide an application example of the present invention below.

This experiment with lasso tool and figure instrument of automatically scratching be contrast object.

common lasso tool need user to fit absorbedly object border describes partitioning boundary.Its Magnetic Lasso Tool is difficult to prove effective when object obscure boundary is clear.The present invention only needs to sketch the contours raw curve along object boundary roughly, and algorithm can be automatically found partitioning boundary accurately.Even if object obscure boundary is clear, also by the raw curve of gradual interpolation, the weight increasing shape constraining obtains the result envisioned.

Current popular software in automatically scratch drawing method, before needing user simply to sketch the contours, background, software carries out global segmentation to entire image automatically.This process employs sketched the contours pixel, before foundation, the statistical model of background color is as segmentation constraint.When front, background color is overlapping, method is difficult to gather effect.The present invention abandons colouring information, then utilizes shape constraining, there are not the problems referred to above.In addition, global approach is split entire image simultaneously, will be difficult to Real-time Feedback segmentation result when processing large-size images.The present invention only calculates near border, and gradually processes piecemeal, even if when picture size is larger, and also can rapid feedback segmentation result.

Fig. 2 is the result of this experiment, and (a) is the image of input and raw curve, and (b) is segmentation result.The former figure of Fig. 2 takes from a part for a width Chinese ancient painting.Original painting with lines for main drafting is shaping, therefore, front, background color is seriously overlapping.In addition, tempering of elapsed time, picture noise is serious, and lines are also no longer clear and legible.From the segmentation result of Fig. 2, even if method of the present invention front, background color is seriously overlapping, when obscure boundary is clear, also can obtain satisfied segmentation result.

Claims (1)

1. a progressive picture dividing method, is characterized in that described method is an iteration, gradual method, and user often adds a raw curve, will trigger the optimizing process of a Shape-based interpolation priori, and every suboptimization all can Real-time Feedback segmentation result; Each iteration comprises the following steps:

Step one, upgrades pattern curve, specifically comprises following content:

(1) raw curve is sketched the contours

Read in image in the user interface, user sketches the contours raw curve with touch-screen, Digitizing plate or mouse along the edge of interested object; All raw curves must be consistent with the start-stop order of pattern curve, all follows counterclockwise or clock-wise order; This order be in step 2 before, the foundation that supposes of the orientation of background;

(2) matched curve

If current iteration belongs to first leg, then raw curve inherently pattern curve; If non-first leg, namely there is partial shape curve, then upgraded pattern curve by the mode of the raw curve of matching and existing pattern curve;

Two class curves form by a series of node; Represent new hyperplasia curve with S, use represent a front iteration, i.e. the pattern curve that formed of t-1 moment, n=1,2 ..., N, N are the number of node; represent that the n-th node is engraved in the position on image when t-1; represent the width of this Nodes curve, namely form the number of this node raw curve used; Assuming that C ^t-1the n-th node corresponding point on S are expressed as q _n, q _ni.e. C ^t-1? the normal at place and the intersection point of curve S; Then current iteration, i.e. the pattern curve C of t ^tthe position of the n-th node on image with the curve width of this Nodes be respectively:

$p_n^t=0.5p_n^{t-1}+0.5q_n$

$w_n^t=w_n^{t-1}+1$

Consider that the raw curve of Freely input is often comparatively mixed and disorderly, regulation: if q _nwith distance be greater than certain threshold value, corresponding point not as merging point consider, then:

$p_n^t=p_n^{t-1}$

$w_n^t=w_n^{t-1}$

(3) surplus capacity is added

If S and C ^t-1possess lamination portion and have surplus capacity, then adding node corresponding for surplus capacity to C ^tin, and at C ^tin local smooth treatment is carried out to the junction of the part that the part of adding as surplus capacity and matching generate;

For the border of object to be split clearly, pattern curve need not be fitted border completely, and gradient information can play conclusive effect in the determination of partitioning boundary; For unsharp border, user needs the mode by adding raw curve more, more adequately designated shape curve, and improves the weight of shape prior;

Step 2, construct and solve energy function, method is as follows:

By in current iteration process, renewal and as surplus capacity add partial shape curve be expressed as C'; Only around C', perform segmentation, before the pixel that the C' that namely adjusts the distance is less than δ is carried out, background class; The value of δ is related to computation complexity, is also related to the search area of real partitioning boundary simultaneously, and in order to balance the two, suggestion δ gets 20 pixels; Specifically comprise following content:

(1) energy function is defined

Involved pixel is expressed as figure G=< ν, ε >, wherein ν represents all pixel set, and ε represents the limit between neighbor; Target is that these pixels are divided into prospect and background, to find the real border through these pixels; Namely x is used _imark each pixel; x _i{ 1,0}, represents prospect and background to ∈ respectively; Definition energy function is as follows:

$E\left(X\right)=\underset{i\&Element;v}{\&Sigma;}{E}_1\left(x_i\right)+\underset{(i,j)\&Element;\mathrm{\&epsiv;}}{\&Sigma;}\&lsqb;{\mathrm{\&omega;}}_{i-j}{E}_2(x_i,x_j)+E_3(x_i,x_j)\&rsqb;$

In formula, E ₁(x _i) be data item, expression is by front, background x _i{ 1,0} marks the cost in each pixel to ∈; E ₂(x _i, x _j), E ₃(x _i, x _j) represent cost when different marks being assigned to adjacent node, wherein, E ₂(x _i, x _j) be shape prior item, E ₃(x _i, x _j) be gradient terms;

(2) data item is defined

The node of graph of left and right side of C' in figure G is expressed as L and R, is expressed as from C' one deck pixel farthest with preset along " right side-prospect, left side-background " on pattern curve start-stop direction, or the supposition of " left side-prospect, right side-background "; Under the supposition of " right side-prospect, left side-background ", setting belongs to pixel be absolute foreground pixel, belong to pixel be absolute background pixel; Definitely, background constraint is in order to avoid figure cuts the minimal cut phenomenon of algorithm; These pixels will not change state after segmentation optimizing process; Segmentation will be right other pixels do before, context marker; E ₁(x _i) be defined as follows:

$\left\{\begin{array}{ccc}{E}_1(x_i=1)=0& E_1(x_i=0)=\mathrm{\&infin;}& i\&Element;{\mathrm{\&Omega;}}_R^{C^{\&prime;}}\\ E_1(x_i=1)=\mathrm{\&infin;}& E_1(x_i=0)=0& i\&Element;{\mathrm{\&Omega;}}_L^{C^{\&prime;}}\\ E_1(x_i=1)=0& E_1(x_i=0)=0& i\&Element;\{v-{\mathrm{\&Omega;}}_R^{C^{\&prime;}}-{\mathrm{\&Omega;}}_L^{C^{\&prime;}}\}\end{array}\right.$

(3) shape prior item is defined

E ₂(x _i, x _j) be defined as:

$E_2(x_i,x_j)=\frac{d_{i-j}^{C^{\&prime;}}}{\mathrm{\&delta;}}|x_i-x_j|$

In formula, represent the distance of the sub-pixel between pixel i and j to C', in energy function, be positioned at E ₂(x _i, x _j) front side ω _i-jthe weight of dynamic adjustments shape prior in cutting procedure, the width of the node that the normal distance of sub-pixel on C' between up-to-date moment pixel i and the j being defined as iteration is nearest; Normal distance on image on certain point and C' between certain node is defined as the distance of this point to this node normal; | x _i-x _j| represent the value of shape prior only on statistical boundary;

(4) gradient terms is defined

E ₃(x _i, x _j) be defined as:

$E_3(x_i,x_j)=\frac{\mathrm{\&lambda;}}{1+D_{i,j}}|x_i-x_j|$

In formula, D _i,j=(r _i-r _j) ²+ (g _i-g _j) ²+ (b _i-b _j) ²represent pixel i, the j distance in rgb space, r _i, g _i, b _irepresent the RGB component of pixel i respectively; λ is a constant, for equilibrium gradient item and initial shape prior item; The value of gradient terms equally only on statistical boundary;

(5) energy function is solved

The present invention adopts figure to cut the minimum value of Algorithm for Solving energy function, obtains local segmentation result; After pattern curve is closed, as sketched the contours and the pattern curve formed according to counter clockwise direction under the supposition of " right side-prospect, left side-background ", after singulation, the part that divided border comprises will be prospect, and other parts divide background into.