CN102592268B - Method for segmenting foreground image - Google Patents

Abstract

The invention discloses a method for segmenting a foreground image. The method comprises the following steps of: obtaining a conspicuity map of an original image by using a central peripheral histogram algorithm; carrying out threshold segmentation on the conspicuity map so as to obtain a rectangle R including a conspicuity object; using an image outside the rectangle R region as a background region; initializing a GrabCut algorithm; and iteratively operating the GrabCut algorithm so as to execute foreground segmentation of the original image. Compared with the prior art, the method disclosed by the invention can be used for increasing the foreground segmentation efficiency.

Description

A kind of method of segmenting foreground image

Technical field

The present invention relates to image processing techniques, particularly relate to a kind of method of segmenting foreground image.

Background technology

Salient region detection and foreground segmentation are two fundamental operations in Computer Image Processing.Wherein, salient region detects and refers to the salient region judging image from picture, and notices the pith of image.Foreground segmentation refers to and allows computing machine judge which is foreground object from a width picture, and which is background object, and is therefrom partitioned into interested prospect critical object.Although the vision system of people can judge salient region and foreground object easily, computing machine is difficult to possess this understandability under inartificial help.If computing machine can be allowed independently to complete foreground segmentation work rapidly, to be convenient to analyze image further, identify, follow the tracks of, understand, compressed encoding etc., and the accuracy extracting result directly will affect the validity of follow-up work, how interested target being split from the background of complexity quickly and efficiently, tool is of great significance.

People have carried out large quantifier elimination on salient region detects, and have summed up the algorithm of a lot of maturation, have mainly contained HC, RC, LC, CA and FT scheduling algorithm, these algorithms can obtain the good Saliency maps of effect (saliencymap) all to a certain extent.And image segmentation algorithm can be roughly divided into 5 classes at present, Boundary algorithm, clustering algorithm, zone algorithm, the partitioning algorithm of segmentation blending algorithm and specific area, in foreground segmentation techniques, mainly contain based on the method for pixel (Pixel-based), based on the method for border (Edge-based) and the method based on region (Region-based).Method based on pixel requires that user specifies prospect or background in single Pixel-level, and therefore workload is very huge.Method based on border allows user around the boundary mapping curve of foreground object, then carries out subsection optimization to this curve, but the curve plotting that user must be careful, still need a large amount of user interactions.The information that method based on region allows user to specify some loose, and use optimized algorithm to extract actual foreground object border.

Summary of the invention

Technical matters to be solved by this invention is, provides a kind of method of segmenting foreground image, and the requirement of reduction operation to user interactions, improves foreground segmentation efficiency.

Technical matters of the present invention is solved by the following technical programs:

A method for segmenting foreground image, is characterized in that, comprises the following steps:

1) central peripheral histogramming algorithm is used to obtain the Saliency maps of original image;

2) Threshold segmentation is carried out to described Saliency maps, obtain the rectangle R comprising conspicuousness object;

3) the image region as a setting using described rectangle Zone R overseas, initialization Grabcut algorithm, iteration runs the foreground segmentation of GrabCut algorithm execution to original image.

Compared with prior art, present invention utilizes the conspicuousness segmentation of image and the relevance of foreground segmentation, the initializes GrabCut algorithm utilizing conspicuousness to split, eliminate the step that user draws rectangle frame initialization GrabCut algorithm in the target image, user zero input can be realized in whole cutting procedure, automatically complete all foreground segmentation actions by computer class, improve the efficiency of foreground segmentation.

Preferably, described step 2) comprise the following steps: utilize predetermined gray threshold to carry out binaryzation to Saliency maps and obtain binary map; Binary map is carried out twice or repeatedly opening operation; UNICOM region maximum in image after measuring and calculating opening operation, selects the rectangle R of certain size and coordinate position, this UNICOM region is included in this rectangle R just.

Described gray threshold is the average gray of Saliency maps.

Preferably, interactive editor's step is also comprised: the partial pixel of original image is set to prospect or background by the input instruction according to user.This preferred version allows user to revise segmentation, makes up the weak point of auto Segmentation.

Accompanying drawing explanation

Fig. 1 is the process flow diagram of the specific embodiment of the invention.

Embodiment

Contrast accompanying drawing below and in conjunction with preferred embodiment, the present invention be explained in detail.

One, the existing ripe image processing techniques that the present invention relates to

In order to help the understanding to technical solution of the present invention, hereafter first the image processing techniques of maturation involved in the present invention is described:

(1) conspicuousness object Examined effect

The Saliency maps of image adopts center-surround algorithm (central peripheral histogramming algorithm) to calculate: the grey level histogram first adding up three Color Channels of two rectangle inside, R ⁱfor the grey level histogram of image in center rectangular area, for the grey level histogram of image in surround rectangular area.Histogrammic fitting degree in center region and surround region is calculated according to formula (1)

${\mathrm{\χ}}^2(R,R_S)=\frac{1}{2}\mathrm{\Σ}\frac{{(R^i-R_S^i)}^2}{R^i+R_S^i}---\left(1\right)$

$f_h(x,I)\∝{\mathrm{\Σ}}_{\left[x^{\′}\right|x\∈R^{*}\left(x^{\′}\right)]}{\mathrm{\ω}}_{{\mathrm{xx}}^{\′}}{\mathrm{\χ}}^2(R^{*}\left(x^{\′}\right),R_S^{*}\left(x^{\′}\right))---\left(2\right)$

${\mathrm{\ω}}_{{\mathrm{xx}}^{\′}}=\exp (-0.5{\mathrm{\σ}}_{x^{\′}}^{-2}{\left|\right|x{-x}^{\′}\left|\right|}^2)---\left(3\right)$

The eigenwert wherein R of x pixel is determined according to formula (2) ^*(x '), represent the center rectangle centered by x ' and periphery rectangle respectively, x ' represents all can become R ^*(x '), center and this R ^*(x '), the pixel of pixel x can be comprised, wherein ω _xx' for Gauss's attenuation function is such as formula shown in (3), ‖ x-x ' ‖ is the Euclidean distance of pixel x apart from center pixel x ', and K is normaliztion constant.

(2) GrabCut foreground segmentation algorithm

GrabCut algorithm improves on the basis of GraphCut algorithm, and wherein GraphCut algorithm is as described below:

Image is seen as figure G={V, a ε }, V is all nodes, and ε is the limit connecting adjacent node.Iamge Segmentation can be used as a binary flag problem, each i ∈ V, has a unique x _i{ prospect is 1 to ∈, and background is that 0} is corresponding with it.All x _iset X can obtain by minimizing Gibbs ENERGY E (X):

$E\left(X\right)=\underset{i\∈V}{\mathrm{\Σ}}{E}_1\left(x_i\right)+\mathrm{\λ}\underset{(i,j)\∈E}{\mathrm{\Σ}}{E}_2(x_i,x_j)$

Same, according to the curve that user draws, we have foreground node collection F and background Node B, unknown node collection U.First use K-Mean method by the node clustering of F, B, calculate the average color of each class, represent the average color set of all prospect classes, background classes is calculate the minor increment of each node i to each prospect class $d_i^B=\min \left|\right|C\left(i\right)-K_n^F\left|\right|,$ With corresponding back pitch from $d_i^B=\min \left|\right|C\left(i\right)-K_n^B\left|\right|,$ Defined formula:

$\left\{\begin{array}{ccc}{E}_1(x_i=1)=0& E_1(x_i=0)=\∞& \∀i\∈F\\ E_1(x_i=1)=\∞& E_1(x_i=0)=0& \∀i\∈B\\ E_1(x_i=1)=\frac{d_i^F}{d_i^F+d_i^B}& E_1(x_i=0)=\frac{d_i^B}{d_i^F+d_i^B}& \∀i\∈U\end{array}\right.$

Front two groups of equatioies ensure that definition inputs consistent with user, and the 3rd group of equation means that the color phase recency with front background decides the mark of unknown point.

E ₂be defined as a function relevant to gradient:

E ₂(x _i，x _j)＝|x _i-x _j|·g(C _ij)

$g\left(\mathrm{\ϵ}\right)=\frac{1}{\mathrm{\ϵ}+1}$ C _ij＝‖C(i)-C(j)‖ ²

E ₂effect be reduce between the pixel that color is close, exist mark change possibility, even if it only occurs on border.

Finally, with E ₁and E ₂as the weights of figure, figure is split, the node division of zone of ignorance in prospect set or background set, just obtain the result of foreground extraction.

GrabCut algorithm improves on the basis of GraphCut: utilize gauss hybrid models (Gaussian MixtureModel, GMM) to replace histogram, gray level image is expanded to coloured image.

In GrabCut algorithm, GMM model is used to set up color image data model.Each GMM can regard the covariance of a K dimension as.Conveniently process GMM, in optimizing process, introduce vectorial k=(k ₁..., k _n..., k _n) as the independent GMM parameter of each pixel, and k _n∈ 1,2 ..., K}, the opacity α on respective pixel point _n=0 or 1.

Gibbs energy function is written as:

E(α，k，θ，z)＝U(α，k，θ，z)+V(α，z)

In formula, α is opacity, α ∈ 1,0}, 0 is background, and 1 is foreground target; Z is image intensity value array,

z＝(z ₁，…，z _n，…，z _N)。

Introduce GMM color data model, its data item may be defined as:

$U(\mathrm{\α},k,\mathrm{\θ},z)=\underset{n}{\mathrm{\Σ}}D({\mathrm{\α}}_n,k_n,\mathrm{\θ},z_n)$

In formula, D (α _n, k _n, θ, z _n)=-logp (z _n| α _n, k _n, θ) and-logn (α _n, k _n),

P () is gaussian probability distribution, and π () is hybrid weight coefficient (Cumulate Sum is constant).So have:

$D({\mathrm{\α}}_n,k_n,\mathrm{\θ},z_n)=-\log \mathrm{\π}({\mathrm{\α}}_n,k_n)+\frac{1}{2}\log \det ({\mathrm{\α}}_n,k_n)+\frac{1}{2}{[z_n-\mathrm{\μ}({\mathrm{\α}}_n,k_n)]}^T\mathrm{\Σ}({\mathrm{\α}}_n,k_n)[z_n-\mathrm{\μ}({\mathrm{\α}}_n,k_n)]$

The parameter of such model is just defined as:

θ＝{π(α，k)，μ(α，k)，∑(α，k)，k＝1，2，…，K}

The level and smooth item of coloured image is,

$V(\mathrm{\α},z)=\mathrm{\γ}\underset{(m,n)\∈c}{\mathrm{\Σ}}[{\mathrm{\α}}_m\≠{\mathrm{\α}}_n]\exp (-\mathrm{\β}{\left|\right|z_m-z_n\left|\right|}^2)$

Wherein, constant beta is determined by following formula: β=(2 < (z _m-z _n) ²>) ^-1, the β obtained by such formula is guaranteed that above formula middle finger is several and suitably changes between high low value.

Two, a specific embodiment of the present invention

The foreground segmentation method of the present embodiment comprises the following steps: use central peripheral histogramming algorithm to obtain the Saliency maps of original image; Threshold segmentation is carried out to described Saliency maps, obtains the rectangle R comprising conspicuousness object; Use the image region as a setting that described rectangle Zone R is overseas, initialization GrabCut algorithm, iteration runs the foreground segmentation of GrabCut algorithm execution to original image.

Hereafter for the concrete processing procedure to original image A, technical scheme of the present invention is further produced:

1) input original image A, in scan image A, the central point of all possible central rectangular R as center-surround histogramming algorithm and periphery rectangle Rs, calculates the grey level histogram R of image in corresponding R, Rs region ⁱand R ⁱ _s, the length and width of the present embodiment rectangle R are 4/3 times that the length and width of 1/5, Rs of image A length and width get R length and width; The eigenwert of each central point is calculated according to previously described formula (1) ~ (3).Histogrammic fitting degree in center region and surround region is calculated according to formula (2); The significance value of x pixel is determined according to formula (2).For meeting R, the significance value of the pixel at Rs center is directly set to 0.

2) choosing certain threshold value, to carry out to Saliency maps the binary map that binaryzation obtains be C, and the threshold value herein chosen is preferably the average gray of Saliency maps B.Twice corrosion expansion i.e. twice opening operation is carried out to binary map C, thus reduces the interference of isolated noise.UNICOM region maximum in measuring and calculating C, selects the rectangle frame of certain size and relevant position, this UNICOM region is included in this rectangle frame just.If this rectangle frame is R.

3) start to use GrabCut algorithm to split to original image A, comprise the following steps:

A) use rectangle 2) in the rectangle R initialization GrabCut algorithm that obtains: the region outside original image rectangle R is considered as background T _bcarry out initialization ternary diagram T (specify prospect, background on the original image and do not determine that the figure that region obtains is exactly ternary diagram T), prospect is set to sky, namely do not determine region T _uget the supplementary set of background

B) for all pixel n ∈ T _g, make opacity α _n=0; N ∈ T _uthere is α _n=1.

C) α is used respectively _n=0 and α _n=1 two set carrys out the GMM model of initialization prospect and background.

4) iteration minimizes, and comprises the following steps:

D) T is tried to achieve _uin the GMM parameter k corresponding to each pixel n _n, k _{n=arg min}d _n(α, k _n, θ, z _n), (this formula represents makes D _n{ α, k _n, θ, z _n) k when getting minimum value _nvalue, lower with).

E) from data Z, GMM parameter θ is obtained, θ=argminU (α, k, θ, z).

F) initial segmentation is obtained with least energy revise ternary diagram T.

G) according to segmentation ternary diagram T f) obtained, from steps d) iteration repeat, until E (α, k, θ, z) convergence, namely after iteration E (α, k, θ, z) compared with last time change very little time stop iteration.

5) if user is to the division of respective pixel satisfaction not, interactive editor can be used to specify indivedual point to be prospect by force, and indivedual point is background, even respective pixel opacity α _n=0 (background) or α _n=1 (prospect), the correction according to user upgrades ternary diagram T accordingly, performs step f).So far display foreground has been split.

Above content is in conjunction with concrete preferred implementation further description made for the present invention, can not assert that specific embodiment of the invention is confined to these explanations.For those skilled in the art, without departing from the inventive concept of the premise, some equivalent to substitute or obvious modification can also be made, and performance or purposes identical, all should be considered as belonging to protection scope of the present invention.

Claims (2)

1. a method for segmenting foreground image, is characterized in that, comprises the following steps:

1) use central peripheral histogramming algorithm to obtain the Saliency maps of original image, comprise the following steps:

Input original image A, in scan image A, the central point of all possible central rectangular R as central peripheral histogramming algorithm and periphery rectangle Rs, calculates the grey level histogram R of image in corresponding R, Rs region ⁱand R ⁱ _s, the length and width of rectangle R are 4/3 times that the length and width of 1/5, Rs of image A length and width get R length and width; The eigenwert of each central point is calculated according to following formula (1) ~ (3), histogrammic fitting degree in Zone R territory and Rs region is calculated according to formula (1), the significance value of x pixel is determined according to formula (2), for meeting R, the significance value of the pixel at Rs center is directly set to 0;

${\mathrm{\&chi;}}^2(R,R_S)=\frac{1}{2}\underset{i}{\mathrm{\&Sigma;}}\frac{{(R^i-R_S^i)}^2}{R^i+R_S^i}---\left(1\right)$

$f_n(x,I)\&Proportional;{\mathrm{\&Sigma;}}_{\left\{x^{\&prime;}\right|x\&Element;R^{*}\left(x^{\&prime;}\right)\}}{\mathrm{\&omega;}}_{{\mathrm{xx}}^{\&prime;}}{\mathrm{\&chi;}}^2(R^{*}\left(x^{\&prime;}\right),R_S^{*}\left(x^{\&prime;}\right))---\left(2\right)$

${\mathrm{\&omega;}}_{{\mathrm{xx}}^{\&prime;}}=\exp (-0.5{\mathrm{\&sigma;}}_{x^{\&prime;}}^{-2}{\left|\right|x-x^{\&prime;}\left|\right|}^2)---\left(3\right)$

Wherein, R ^*(x'), represent the center rectangle centered by x ＇ and periphery rectangle respectively, x ＇ represents all can become R ^*(x'), center and this R ^*(x'), the pixel of pixel x can be comprised, wherein ω _xx＇ is Gauss's attenuation function;

2) Threshold segmentation is carried out to described Saliency maps, obtains the rectangle S comprising conspicuousness object, comprise the following steps:

It is C that the average gray choosing Saliency maps carries out to Saliency maps the binary map that binaryzation obtains, twice corrosion expansion is carried out to binary map C, thus reduce the interference of isolated noise, calculate connected region maximum in the binary map C after corrosion is expanded, select the rectangle frame S of certain size and relevant position, make this connected region just be included in this rectangle frame;

3) use the extra-regional image of described rectangle S region as a setting, initialization GrabCut algorithm, iteration runs the foreground segmentation of GrabCut algorithm execution to original image.

2. the method for segmenting foreground image according to claim 1, is characterized in that, also comprises interactive editor's step: the partial pixel of original image is set to prospect or background by the input instruction according to user.