A kind of image partition method based on hypergraph analysis of spectrum
Technical field
The invention belongs to computer vision and technical field of image processing, especially relate to a kind of image partition method based on hypergraph analysis of spectrum.
Background technology
At computer vision field, Iamge Segmentation (Segmentation) refers to process digital picture being subdivided into multiple image region.The object of Iamge Segmentation is the representation simplifying or change image, is the basic work of Image processing and compute machine visual field, image is easier to understand and analyzes.In reality, although people can be easy to image can be divided into coherent region, for being very difficult with computer vision system.Although there is a lot of dividing method to be suggested, due to natural image diversity and polysemy, the performance of segmentation can't meet actual demand.
At present, the quantity of image partition method is very many, and wherein, the method based on graph theory causes the extensive concern of scholars in the past for 30 years.The main thought of Graph-theoretical Approach is that image mapped is become weighted graph, image pixel is seen the summit of mapping, relation between adjacent pixels sees the limit of mapping, similarity between adjacent pixels regards the weights on limit as, according to the weights design energy function on limit, complete the segmentation to figure by minimization of energy function, thus realize Iamge Segmentation.
Dividing method based on graph theory is obvious to advantage during Image Segmentation Using:
1) graph theory is a research relatively morning and full-fledged subject, has good Fundamentals of Mathematics.For certain problem, multiple method in graph theory, is had to solve;
2) closely similar between image and figure.
After image mapped is figure, the various theory in graph theory and mathematical tool just can be utilized to carry out Iamge Segmentation.The dividing method based on graph theory conventional at present comprises minimal cut (being commonly referred to figure to cut), normalized cut etc.
For figure segmentation method, its advantage is mainly:
1) split under the framework of global optimum, ensure that the globally optimal solution of energy function;
2) make use of pixel grey scale information and the zone boundary information of image, segmentation effect is good simultaneously;
3) user interactions is simple and convenient, only need mark a small amount of Seed Points in target internal and background area, also not be strict with, and automatically determine Seed Points by preprocess method to the particular location of Seed Points, also can allow the robotization of figure segmentation method.
Cutting with figure and compare, there are following 2 deficiencies in normalized cut:
1) do not embed unitary (Unary) item, as the priori of respective figure node, being equivalent to all nodes is all zero priori;
2) need the generalized eigenvector calculating large matrix, although take complexity braking measure, calculated amount is still very large.
But it is maximum that normalized cut can not only meet segmentation result similar degree in the class, distinctiveness ratio between class can be made maximum, therefore, it more easily isolates the Small object object in image, and this is most important for Iamge Segmentation simultaneously.
But for iconic model, only based on traditional structure of pixel between two opening relationships (limit), the graph model of formation obviously cannot complex relationship between picture engraving region, the result of Iamge Segmentation often and unsatisfactory.
Summary of the invention
For solving the problem, the invention discloses a kind of image partition method based on hypergraph analysis of spectrum, passing through hypergraph model, super-pixel point is utilized to replace pixel, utilize the high-order coupled relation between super-pixel, the relevance between reflection image regional, reaches optimum segmentation result.
In order to achieve the above object, the invention provides following technical scheme:
Based on an image partition method for hypergraph spectrum segmentation, comprise the steps:
Image semantic classification step: based on given image, adopt at least two kinds of over-segmentation methods to carry out over-segmentation to image respectively, wherein, image is divided into several subregions by often kind of over-segmentation method, using every sub regions as a super-pixel, thus obtain multiple over-segmentation super-pixel set of image;
Image superpixel combining step: based on the multiple image over-segmentation super-pixel set obtained in Image semantic classification step, sought common ground between two by the mode of iteration, finally obtains merging super-pixel set;
Hypergraph model generation step: utilize the super-pixel merged in super-pixel set to represent hypergraph summit, and build super limit with the super-pixel in the set of often kind of over-segmentation super-pixel and the relevance merged between super-pixel set, add up the weight of the over-segmentation super-pixel relevant to each super limit the weight of sum as super limit, thus form hypergraph model;
Hypergraph spectral clustering step: based on the hypergraph model generated, by hypergraph Spectral Clustering by super-pixel cluster, thus obtain the segmentation result of image.
As a preferred embodiment of the present invention, the over-segmentation method in described Image semantic classification step comprise Kmeans dividing method, Meanshift dividing method, based on the Hierarchical Segmentation method of contour detecting and multiple dimensioned normalization figure segmentation method.
As a preferred embodiment of the present invention, described in described image superpixel combining step, the mode of iteration seeks common ground between two and is expressed as: I=((((S
1∩ S
2) ∩ S
3) ∩ S
4) ... ∩ S
m), wherein, I represents the set of merging super-pixel, S
1s
mrepresent the over-segmentation super-pixel set generated by M kind over-segmentation method respectively.
As a preferred embodiment of the present invention, the process building super limit in described hypergraph model generation step is: if the super-pixel merged in super-pixel set is a part for super-pixel in the set of over-segmentation super-pixel, then form a super limit, distribute numerical value 1 to super limit incidence matrix simultaneously, otherwise distribute numerical value 0.
As a preferred embodiment of the present invention, describedly by hypergraph Spectral Clustering by the process of super-pixel cluster be: after the eigenwert solving hypergraph Laplacian Matrix and proper vector, utilize kmeans clustering algorithm to be separated by hypergraph optimum.
Compared with prior art, the present invention adopts the advantage of above technical scheme to be:
First, the present invention using the region after image over-segmentation as super-pixel, and using super-pixel as summit, relation between the super-pixel obtained with multiple different over-segmentation is to build super limit, form hypergraph model, instead of the method that traditional opening relationships of pixel between two (limit) forms graph model, thus effectively reflect the high-order coupled relation between pixel, consider the relevance between image regional, the region consistency of segmentation result and edge accuracy are all had clear improvement.
Secondly, experimental result shows, the segmentation result that the present invention obtains is more accurate than existing methods, for successive image treatment and analysis provides good basis, and can widespread use and computer vision and image processing field.
Accompanying drawing explanation
Fig. 1 is the image partition method flow chart of steps based on hypergraph analysis of spectrum provided by the invention.
Fig. 2 is the schematic illustration of hypergraph model.
Fig. 3 is the schematic illustration of hypergraph incidence matrix.
Fig. 4 is the schematic diagram that image mapped becomes hypergraph model.
Fig. 5 is the process schematic of process real image of the present invention.
Fig. 6 adopts the segmentation result schematic diagram of the inventive method in Berkeley University BSDS300 image data base;
Wherein (a) is pending image, and the edge display figure that (b) is image segmentation result, (c) is image segmentation result pseudo-color processing figure.
Embodiment
Below with reference to specific embodiment, technical scheme provided by the invention is described in detail, following embodiment should be understood and be only not used in for illustration of the present invention and limit the scope of the invention.
As shown in Figure 1, the present invention includes following steps:
1, Image semantic classification step: based on given image, adopt at least two kinds of over-segmentation methods to carry out over-segmentation to image respectively, image is divided into several subregions by described over-segmentation method, and every sub regions as a super-pixel, thus obtains super-pixel set.Owing to have employed multiple over-segmentation method, therefore this step obtains the set of multiple over-segmentation super-pixel.Above-mentioned over-segmentation method comprises: Kmeans dividing method, Meanshift dividing method, Hierarchical Segmentation method and multiple dimensioned normalization figure segmentation method etc. based on contour detecting.
2, image superpixel combining step: several image superpixel set of obtaining based on aforementioned multiple over-segmentation method (under be called the set of over-segmentation super-pixel), seek common ground between two in a kind of mode of iteration, and obtain final amalgamation result, be expressed as the super-pixel set (under be called merge super-pixel set) of image.The mode of above-mentioned iteration can show as: first seek common ground to the set of two kinds of over-segmentation super-pixel, then tries to achieve result and seeks common ground with the third over-segmentation super-pixel set, by that analogy.Specifically, generating over-segmentation super-pixel set expression by M kind over-segmentation method is S
1s
m, the merging super-pixel set I obtained after being sought common ground between two by iterative manner is expressed as form:
I=((((S
1∩S
2)∩S
3)∩S
4)…∩S
M)
3, hypergraph model generation step: utilize the super-pixel merged in super-pixel set I to represent hypergraph summit, and build super limit with the super-pixel in the set of often kind of over-segmentation super-pixel and the relevance merged between super-pixel set I, form hypergraph model.
We know, a hypergraph model G=(V, E, W) gathers E and super limit weight matrix W by vertex set V, super limit to form.Fig. 2, Fig. 3 are hypergraph model and incidence matrix schematic illustration thereof, and wherein Fig. 2 lists three super limits and 6 summits, super limit e
1by vertex v
1, v
2, v
3three some compositions, e
2by v
2, v
4two some compositions, e
3by v
5, v
6two some compositions; Incidence matrix H shown in Fig. 3 shows the incidence relation on super limit and summit.Each super limit e
ithere is a weight w (e
i), W=diag (w (e
1), w (e
2) ...).The annexation of hypergraph G is represented and is | V| × | the incidence matrix H of E| is defined as follows:
According to incidence matrix H, the degree of each vertex v ∈ V is expressed as:
d(v)=∑
e∈Ew(e)H(v,e);
The degree of each super limit e ∈ E is expressed as:
δ(e)=∑
v∈VH(v,e)。
Be attached in the present invention, when generating hypergraph model, using the super-pixel in merging super-pixel set I as hypergraph summit.Suppose often kind of over-segmentation super-pixel S set
1s
m, wherein often kind of over-segmentation super-pixel S set
icomprise N number of super-pixel, for each super-pixel R
j, wherein j=1 ... N, N are the number of super-pixel in the set of over-segmentation super-pixel; If the super-pixel k merged in super-pixel set I is the part in super-pixel R, then form a super limit, distribute numerical value 1 to hypergraph incidence matrix simultaneously, otherwise distribute numerical value 0.
Fig. 4 is the schematic diagram that image mapped becomes hypergraph model, and merging super-pixel k in super-pixel set I is over-segmentation super-pixel S set
isuper-pixel R
iand R
jin a part, i=1 ... N, because which form wherein super limit e, R
iand R
jtwo super-pixel forming super limit e.
According to above-mentioned super limit generation type iterative computation, thus build hypergraph incidence matrix.Each super-pixel R
jdistribute corresponding weight w (R
j), then using the weight of the weight sum of the over-segmentation super-pixel relevant to each super limit as super limit:
Hypergraph model is formed according to above-mentioned hypergraph incidence matrix and super limit weight.
4, hypergraph spectral clustering step: based on the hypergraph model generated in above-mentioned steps, utilizes the thought of hypergraph spectral clustering that super-pixel is polymerized to different classes, thus obtains the segmentation result of image.Concrete, the present invention is based on space invariance to define hypergraph Spectral Clustering, the cluster objective function that its space invariance is mainly reflected in definition does not consider the size on super limit, only considers annexation and the weight allocation on super limit.The hypergraph normalization framework of the present invention's definition:
Wherein, f is prediction class, and Ω (f) is hypergraph normalized energy item, R
empf () is empirical loss item, λ is balance parameters and λ > 0.In practical problems, for non-supervisory image, supervision message cannot use, and therefore, the present invention does not consider empirical loss item R
emp(f), only consider hypergraph normalized energy item Ω (f), it is defined as:
Wherein, u, v are two different summits in hypergraph respectively, H (u, e), H (v, e) is hypergraph incidence matrix respectively, and δ (e) is the degree of super limit e, f (u), f (v) are identical with f implication, represent prediction class; D (v), d (u) are the degree on hypergraph summit respectively.
Definition
hypergraph Laplacian Matrix Δ=I-Θ, then hypergraph normalized energy item is finally defined as:
Ω(f)=f
TΔf
Wherein, T is transposition; D
vhypergraph Vertex Degree matrix, D
esuper edge degree matrix.
Utilize the thought of hypergraph spectral clustering that super-pixel is polymerized to different classes, obtain image segmentation result.This process is specially: the eigenwert and the proper vector that solve hypergraph Laplacian Matrix, then utilizes kmeans clustering algorithm to be separated by hypergraph optimum, thus obtains final segmentation result.
Fig. 5 is the procedure chart that the present invention progressively processes a secondary real image, and Fig. 6 is the segmentation result of the inventive method in Berkeley University BSDS300 image data base.Can find from segmentation result, the edge of natural image is all showed well, and maintains the consistance in region.
Table 1 is the analysis and assessment of the segmentation result of the inventive method in Berkeley University BSDS300 image data base, wherein first three plants as individually adopting Kmeans method, Meanshift method and Ncut method directly carry out splitting the assessed value obtaining result, rear four kinds be adopt method of the present invention to obtain the assessed value of result (wherein the 4th kind of method is for adopting Kmeans method, Meanshift method carries out Image semantic classification, Lung biopsy carries out Image semantic classification for adopting Kmeans method and Ncut method, 6th kind of method is for adopting Meanshift method and Ncut method row Image semantic classification, 7th kind of method is for adopting Kmeans method, Meanshift method and Ncut method carry out Image semantic classification).Wherein often open image all to cut through different staff work points, on average often opening image can have 6 ground-truths.
We assess segmentation result from three standards: the distance measure (VoI), the global coherency error (GCE) that are blue moral index (PRI) of probability, variation respectively.As can be seen from Table 1, generally speaking, segmentation result of the present invention is substantially better than additive method under three standards.Experimental result shows, as long as it is suitable to choose K value, region consistency and the edge accuracy of segmentation result of the present invention are improved significantly.
Table 1
NO. |
Method |
PRI |
VoI |
GCE |
1 |
Kmeans① |
0.7523 |
2.4929 |
0.2422 |
2 |
Meanshift② |
0.7958 |
2.4802 |
0.1888 |
3 |
Ncut③ |
0.7429 |
2.6013 |
0.2232 |
4 |
Hypergraph with①and② |
0.7688 |
2.4165 |
0.2055 |
5 |
Hypergraph with①and③ |
0.7448 |
2.5852 |
0.2177 |
6 |
Hypergraph with②and③ |
0.7375 |
2.5728 |
0.1915 |
7 |
Hypergraph with①②③ |
0.8146 |
2.4645 |
0.1779 |
Technological means disclosed in the present invention program is not limited only to the technological means disclosed in above-mentioned embodiment, also comprises the technical scheme be made up of above technical characteristic combination in any.It should be pointed out that for those skilled in the art, under the premise without departing from the principles of the invention, can also make some improvements and modifications, these improvements and modifications are also considered as protection scope of the present invention.