CN103413307A - Method for image co-segmentation based on hypergraph - Google Patents

Abstract

The invention relates to a method for image co-segmentation based on a hypergraph. The method comprises the following steps that two images with the similar or the same foregrounds are read in; segmentation process is conducted on the input images through the Mean-shift algorithm; statistical operation is conducted on a color histogram H of the images through a histogram statistical method; the similarity of adjacent sub blocks in one image and the similarity of similar sub blocks in the different images are calculated; a similarity matrix of nodes and a similarity matrix of hyperedges are calculated, the proper value and the proper vector of the Laplacian matrix of the hypergraph are calculated and the foregrounds and backgrounds are segmented. According to the method, calculating speed of segmentation is improved, the foregrounds are well related, the backgrounds are well related, more relative information of the sub blocks is kept, and the segmenting effect is improved.

Description

A kind of image based on hypergraph is divided into segmentation method

Technical field

The present invention relates to a kind of image based on hypergraph and be divided into segmentation method, belong to technical field of image processing.

Background technology

Image is divided into the problem of cutting and refers to from two width with identical or similar target or multiple image, being partitioned into the problem of target.For without supervision, automatically image is divided into segmentation method that a lot of work has been arranged, such as being divided into, the image based on histogram similarity cuts algorithm (C.Rother, T.Minka, A.Blake and V.Kolmogorov.Cosegmentation of image pairs by histogram matching:Incorporation a global constraint into MRFs.CVPR, 2006; L.Mukherjee, V.Singh, and C.Dyer.Half-integrality based algorithms for cosegmentation of images.In CVPR, 2009).Above image is divided into and cuts algorithm and can to being divided into, cut processing to image.But when the similarity of image was calculated, based on the color histogram information of image, when prospect and background were similar on color, its segmentation result there will be larger error, caused prospect background to distinguish merely.Simultaneously, above algorithm adopts the dividing method based on markov random file, need to solve the optimization method of a complexity, therefore is difficult to fast image be cut to processing to being divided into.

At image, be divided in segmentation method, in order to introduce more prior imformation to obtain more accurate segmentation result, need first image to be carried out the extraction of different characteristic, as Vicente S, Rother C, Kolmogorov V.Object cosegmentation.CVPR, 2011, the method is extracted the size of image, color, texture, edge, shape, the series of features such as SIFT are learnt.But the method has increased the complexity of feature extraction, increased simultaneously the dimension of characteristics of image, cause computation complexity to increase, splitting speed reduces.

In order to solve the complexity that reduces optimization problem, at first image is carried out to the over-segmentation processing, (Kim E for example, Li H, Huang X.A hierarchical image clustering cosegmentation framework.CVPR, 2012), at first the method carries out the over-segmentation processing to image, take and surpass pixel and carry out composition as node, and by the LAB color of super pixel, HOG and SURF latent structure figure are cut apart.But the method adopts traditional simple graph map image, can only mean two correlationships between node, and be difficult to express correlationship complicated between image, make final segmentation effect descend.

Summary of the invention

The objective of the invention is to cut algorithm in order to solve based on being divided into of traditional simple graph, the composition complexity, solve the problem that computation complexity is high, propose a kind of can be more efficiently to image to being divided into the method for cutting processing.

The present invention proposes a kind of image based on hypergraph and be divided into segmentation method, the method comprises the steps:

Step 1, image reading;

Read in the image that two width have similar or identical prospect;

Step 2, image are divided into and cut;

Adopt the Mean-shift algorithm to carry out the over-segmentation processing to input picture, the label of the affiliated piecemeal of each pixel in rear image cut apart in record;

Step 3, block similarity matching judgement;

To image block, adopt its color histogram of statistics with histogram method statistic H; To the piecemeal i in image 1 and the piecemeal j compute histograms Chi distance in image 2:

$d(H_{1i},H_{2j})=\underset{k}{\mathrm{\Σ}}\frac{{(H_{1i,k}-H_{2j,k})}^2}{{(H_{1i,k}+H_{2j,k})}^2}$

H wherein _{Li, k}The k component of the color histogram of piecemeal i in presentation video l; The piecemeal that histogram Chi distance is less than to threshold value is defined as similar piecemeal;

Step 4, block similarity matching calculating;

For the piecemeal in same width image, two piecemeals that definition minute interblock contains 4 adjacent pixels are in abutting connection with piecemeal; Calculate adjacent minute interblock in same width image, and the similarity w of similar minute interblock in different images _Ij:

$w_{\mathrm{ij}}=\left\{\begin{array}{cc}\frac{1}{1+\mathrm{\α}{\left|\right|C_i-C_j\left|\right|}^2}& ,i\≠j\\ 0& ,i=j\end{array}\right.$

Wherein, w _IjMean piecemeal i, the similarity between j, C _iMean the color feature vector of piecemeal i, α is the weighting parameter, α>1; Across the similarity between image block, be multiplied by weighting factor λ, λ>1;

Step 5, hypergraph partitioning;

For the hypergraph of step 4 gained, the degree matrix on computing node and super limit:

$d\left(v_j\right)={\mathrm{\Σ}}_{e_i\∈E}\mathrm{\ω}\left(e_i\right)h(v_j,e_i)$

$\mathrm{\δ}\left(e_i\right)={\mathrm{\Σ}}_{v_j\∈V}h(v_j,e_i)$

V wherein _jMean hypergraph node j, e _iMean super limit i, h (v _j, e _i) mean the corresponding relation on node and limit, ω (e _i) the super limit e of expression _iWeights, δ (e _i) the super limit e of expression _iDegree, d (v _j) expression node v _jDegree;

According to the hypergraph partitioning algorithm, calculate the Laplacian Matrix of hypergraph

$\mathrm{\Δ}=I-D_v^{-1/2}{\mathrm{HWD}}_e^{-1}{H}^T{D}_v^{-1/2}$

Wherein △ means the Laplacian Matrix of hypergraph, I representation unit diagonal matrix, D _vMean the diagonal matrix of hypergraph node degree, H means hypergraph node and limit corresponding relation matrix, and W means super limit weight matrix, D _eThe matrix that means the super limit of hypergraph degree;

Calculate eigenwert and the proper vector of hypergraph Laplacian Matrix, corresponding each piecemeal label of every one dimension of minimum non-zero eigenwert characteristic of correspondence vector wherein, if be greater than 0, the piecemeal of this label belongs to prospect, is less than 0, belongs to background.

Beneficial effect:

(1) traditional image is divided into and cuts algorithm and carry out cluster based on pixel scale, therefore usually adopting the gray-scale value of pixel or color value processes as feature, can not well express the relevant information of image, and at first the inventive method adopts Mean-shift to carry out the over-segmentation processing to image, using the piecemeal of over-segmentation as cutting unit, itself introduced so the conforming information of regional area of image, simultaneously blocking characteristic is than pixel horn of plenty more; Simultaneously, adopt Mean-shift to carry out the over-segmentation processing to image, the piecemeal quantity obtained is far smaller than the quantity of pixel, makes the computation complexity decrease of final optimization pass equation solution, has improved the computing velocity of cutting apart.

(2) for by all segment fusions in two width images in same hypergraph, the present invention judges for a minute interblock similarity in different images, adjacent piecemeal in similar piecemeal and same image is calculated to similarity, can well the similar piecemeal in different images be connected like this, give larger weights, and by dissimilar piecemeal separately; Give larger weighting for simultaneously similar minute interblock similarity weights between image, make like this between prospect and prospect, background and background and better connect, improve the effect of cutting apart; Adopt the set of super limit as similar piecemeal, make the expression of hypergraph more simple compared to simple graph, and kept more piecemeal relevant information, improved the effect of cutting apart.

The accompanying drawing explanation

Fig. 1 is that image of the present invention is divided into the process flow diagram cut;

Fig. 2 is the right Mean-shift over-segmentation result of image;

Fig. 3 is the hypergraph organigram;

Fig. 4 is divided into to cut the result demonstration.

Embodiment

Below in conjunction with the embodiment of accompanying drawing to the inventive method, elaborate.

A kind of image based on hypergraph is divided into segmentation method, the method at first to image to carrying out the Mean-shift over-segmentation, using the cut zone piecemeal that obtains as the hypergraph node; Then utilize the color characteristic of piecemeal to calculate the similarity of minute interblock, using similar node set and adjacent node set as super limit and calculate its weights, construct hypergraph; Finally utilize the approximate data based on analysis of spectrum to solve the hypergraph partitioning problem, obtain right being divided into of image and cut result, particular flow sheet as shown in Figure 1.

A kind of image based on hypergraph is divided into segmentation method, and its specific implementation process is as follows:

Step 1, image reading;

Read in the image that two width have similar or identical prospect;

Step 2, image are divided into and cut;

Adopt Mean-shift algorithm (Comaniciu D and Meer P, Mean shift:A robust approach toward feature space analysis.IEEE Transactions on Pattern Analysis and Machine Intelligence) image read in step 1 is carried out to the over-segmentation processing, the label of the affiliated piecemeal of each pixel in rear image cut apart in record, and the over-segmentation result as shown in Figure 2.

Step 3, block similarity matching judgement;

Owing in step 2, the over-segmentation pre-service of image being based on the colouring information of image, so the consistance of piecemeal self color characteristic is better, and the otherness of minute interblock color characteristic is larger, therefore adopt color characteristic can be well the similarity of piecemeal to be judged, its color histogram of employing statistics with histogram method statistic H; Chi distance to the piecemeal i in image 1 and the piecemeal j compute histograms in image 2:

$d(H_{1i},H_{2j})=\underset{k}{\mathrm{\Σ}}\frac{{(H_{1i,k}-H_{2j,k})}^2}{{(H_{1i,k}+H_{2j,k})}^2}$

H wherein _{Li, k}The k component of the color histogram of piecemeal i in presentation video l; The piecemeal that Histogram correlation is less than to threshold value is defined as similar piecemeal, and getting threshold value here is 0.2.

Step 4, block similarity matching calculating;

For the piecemeal in same sub-picture, two piecemeals that definition minute interblock contains 4 adjacent pixels are in abutting connection with piecemeal.Calculate adjacent minute interblock in same sub-picture, and the similarity of similar minute interblock in different images:

$w_{\mathrm{ij}}=\left\{\begin{array}{cc}\frac{1}{1+\mathrm{\α}{\left|\right|C_i-C_j\left|\right|}^2}& ,i\≠j\\ 0& ,i=j\end{array}\right.$

Wherein, w _IjMean piecemeal i, the similarity between j, C _iMean the color feature vector of piecemeal i, α is the weighting parameter, gets α=5; Across the similarity between image block, be multiplied by weighting factor λ, got λ=2.

The hypergraph building method, i.e. the calculating on super limit diagram, as shown in Figure 3.

Step 5, hypergraph partitioning;

For the matrix of step 4 gained, the degree matrix on computing node and super limit:

$d\left(v_j\right)={\mathrm{\Σ}}_{e_i\∈E}\mathrm{\ω}\left(e_i\right)h(v_j,e_i)$

$\mathrm{\δ}\left(e_i\right)={\mathrm{\Σ}}_{v_j\∈V}h(v_j,e_i)$

V wherein _jMean hypergraph node j, e _iMean super limit i, h (v _j, e _i) mean the corresponding relation on node and limit, ω (e _i) the super limit e of expression _iWeights, δ (e _i) the super limit e of expression _iDegree, d (v _j) expression node v _jDegree

Adopt hypergraph partitioning algorithm (Zhou D, Huang J, Scholkopf B.Learning with hypergraphs:Clustering, classification, and embedding.Advances in neural information processing systems, 2007), calculate the Laplacian Matrix of hypergraph

$\mathrm{\Δ}=I-D_v^{-1/2}{\mathrm{HWD}}_e^{-1}{H}^T{D}_v^{-1/2}$

Wherein △ means the Laplacian Matrix I representation unit diagonal matrix D of hypergraph _vThe diagonal matrix H that means the hypergraph node degree means that hypergraph node and limit corresponding relation matrix W mean super limit weight matrix D _eThe matrix that means the super limit of hypergraph degree.

Calculate eigenwert and the proper vector of hypergraph Laplacian Matrix, corresponding each piecemeal label of every one dimension of minimum non-zero eigenwert characteristic of correspondence vector wherein, if be greater than 0, the piecemeal of this label belongs to prospect, is less than 0, belongs to background.Be divided into and cut result, as shown in Figure 4.When image to the situation with a plurality of similar targets under, the matrix column vector of the composition of K proper vector is adopted to the K-means clustering algorithm, the corresponding segmentation result of cluster result.

The present invention is not limited only to above embodiment, everyly utilizes mentality of designing of the present invention, does the design of some simple change, within all should counting protection scope of the present invention.

Claims (1)

1. the image based on hypergraph is divided into segmentation method, it is characterized in that, comprises the steps:

Step 1, image reading;

Read in the image that two width have similar or identical prospect;

Step 2, image are divided into and cut;

Adopt the Mean-shift algorithm to carry out the over-segmentation processing to input picture, the label of the affiliated piecemeal of each pixel in rear image cut apart in record;

Step 3, block similarity matching judgement;

To image block, adopt its color histogram of statistics with histogram method statistic H; To the piecemeal i in image 1 and the piecemeal j compute histograms Chi distance in image 2:

$d(H_{1i},H_{2j})=\underset{k}{\mathrm{\Σ}}\frac{{(H_{1i,k}-H_{2j,k})}^2}{{(H_{1i,k}+H_{2j,k})}^2}$

H wherein _{Li, k}The k component of the color histogram of piecemeal i in presentation video l; The piecemeal that histogram Chi distance is less than to threshold value is defined as similar piecemeal;

Step 4, block similarity matching calculating;

For the piecemeal in same width image, two piecemeals that definition minute interblock contains 4 adjacent pixels are in abutting connection with piecemeal; Calculate adjacent minute interblock in same width image, and the similarity w of similar minute interblock in different images _Ij:

$w_{\mathrm{ij}}=\left\{\begin{array}{cc}\frac{1}{1+\mathrm{\α}{\left|\right|C_i-C_j\left|\right|}^2}& ,i\≠j\\ 0& ,i=j\end{array}\right.$

Wherein, w _IjMean piecemeal i, the similarity between j, C _iMean the color feature vector of piecemeal i, α is the weighting parameter, α>1; Across the similarity between image block, be multiplied by weighting factor λ, λ>1;

Step 5, hypergraph partitioning;

For the hypergraph of step 4 gained, the degree matrix on computing node and super limit:

$d\left(v_j\right)={\mathrm{\Σ}}_{e_i\∈E}\mathrm{\ω}\left(e_i\right)h(v_j,e_i)$

$\mathrm{\δ}\left(e_i\right)={\mathrm{\Σ}}_{v_j\∈V}h(v_j,e_i)$

V wherein _jMean hypergraph node j, e _iMean super limit i, h (v _j, e _i) mean the corresponding relation on node and limit, ω (e _i) the super limit e of expression _iWeights, δ (e _i) the super limit e of expression _iDegree, d (v _j) expression node v _jDegree;

According to the hypergraph partitioning algorithm, calculate the Laplacian Matrix of hypergraph

$\mathrm{\Δ}=I-D_v^{-1/2}{\mathrm{HWD}}_e^{-1}{H}^T{D}_v^{-1/2}$

Wherein △ means the Laplacian Matrix of hypergraph, I representation unit diagonal matrix, D _vMean the diagonal matrix of hypergraph node degree, H means hypergraph node and limit corresponding relation matrix, and W means super limit weight matrix, D _eThe matrix that means the super limit of hypergraph degree;

Calculate eigenwert and the proper vector of hypergraph Laplacian Matrix, corresponding each piecemeal label of every one dimension of minimum non-zero eigenwert characteristic of correspondence vector wherein, if be greater than 0, the piecemeal of this label belongs to prospect, is less than 0, belongs to background.

Images