CN103578096A - Image segmentation algorithm - Google Patents

Abstract

Disclosed is an image segmentation algorithm adopting the graph theory technology. Pixels of images are expressed by peaks of graphs. A minimum spanning tree is generated, edges of the tree are successively removed to keep a spanning forest according to morphology characteristics, and trees of the spanning forest correspond to parts of the images. The selection of the edges needing removing can depend on an energy function of the trees and energy functions of trees generated after the edges are removed.

Description

A kind of image segmentation algorithm

Technical field

The present invention relates to a kind of image processing techniques, particularly a kind of partitioning algorithm that uses the view data of graph-theory techniques.

Background technology

Be segmented in the many application that comprise noise reduction and image and video data compression of great use, and previously proposed multiple image segmentation algorithm.These algorithms comprise threshold value, watershed algorithm, Morphological Scale-Space conversion or screening, Region Segmentation and merging, cluster, frequency domain technique and the technology that comprises estimation.Yet seldom (if any) can be used the low high-quality level that generates many component images to medium processing resource to cut apart in these algorithms.

The present invention uses some concepts from branch of mathematics (being usually said graph theory).Below that in graph theory, the cylinder of term used will be described.

Figure is comprised of the point (being called summit) that can couple together by line (being called limit).The example of figure has been shown in Fig. 1.

The degree on summit is the quantity on connected limit.In Fig. 1, the scope of Vertex Degree is 0 to 3.If summit degree of having 1, it is called as leaf; The figure of Fig. 1 has three leaves.

Limit and summit all can have numeral or the weight being associated with them.These weights can have some physical significances; For example, in vertex representation cities and towns and limit represent in the figure of road, limit power (edgeweight) can represent link length.

The subgraph of figure G is that its summit is the figure of the subset on summit and the subset on the limit that its limit is G of G.If subgraph has all summits of G, it generates (span) G.Path in G is that the set of the different edge on summit is all shared on every one side with front one side.For more precise definition, can be referring to Bollobas, Bela.Graph theory-an introductory course.Springer-Verlag, New York, 1979.

Tree is wherein from any given summit, to take office the figure that just in time there is a paths on its summit what.Forest is one group of separated tree.Spanning tree or the spanning forest of figure G are tree or the forests as the spanning subgraph of G.In the figure of limit weighting, the minimum spanning tree of figure (MST) (being also known as the shortest or economic spanning tree) is the spanning tree that makes limit power summation minimum in this tree.

Exist for searching the algorithm known of the minimum spanning tree of limit weighted graph.A Kruskal algorithm, it maintains one group of Local Minimum Spanning Tree and repeats to add minimal weight (or the lightest) limit of its summit in different spanning trees.Another kind is Prim algorithm, and it and by repeating to add in this tree the lightest limit that connects this tree to also non-existent summit, then adds this summit in this tree to and set up tree from single summit.In greater detail in Fig. 4 and Fig. 5, Fig. 5 shows the minimum spanning tree of Fig. 4 below.

Consider now graph theory to be applied in image.The summit of figure can be used for representing pixel, and limit can be used for representing the adjacency of pixel.In this article, if pixel just in time above one other pixel, below, left side or right side, two pixel adjacency, therefore have the limit that connects them.This is that 4 of adjacency connects definition; Can also use 6 connections or 8 to connect definition.By 4 connections, define, can pass through figure presentation video as shown in Figure 2.The degree on each summit of the borderline pixel of presentation video is not 4.Note, in the word using in graph theory " limit " and image or the conceptual relation on the limit in image boundary very little.

Limit power in this figure can be used for representing some tolerance (measure) poor between neighbor.For example, in luminance picture, Bian Quanke is defined as two absolute differences between brightness value.

Fig. 3 shows the exemplary 5x4 luminance picture of the pixel value having as shown in the figure, and Fig. 4 shows the figure that represents this image, wherein, has distributed the limit power that equals the absolute difference between adjacent pixel values.

MST can be used as the basis of image segmentation algorithm.Can find out, if remove a limit from tree, will form two trees, every tree in this image range will be described the vertex subset connecting, that is, and and the part of image.Therefore,, for image being divided into N part, need to from MST, remove N-1 bar limit.Previously having proposed to select the limit of removing is only N-1 bar weight limit (or the heaviest) limit in MST.For example, for the image being represented by Fig. 5 being divided into 2 parts, need to remove the limit of weighting 4, generate cutting apart shown in Fig. 6.

If need 3 parts, also to remove the second heavy limit (weight is 3); Generate cutting apart shown in Fig. 7.

Yet, find that this algorithm has many shortcomings.

Summary of the invention

The improvement algorithm that the object of the present invention is to provide a kind of image to cut apart.

According to a first aspect of the invention, the algorithm that provides a kind of image to cut apart, wherein, the pixel of image is by the vertex representation of figure, the limit in abutting connection with by figure of pixel represents, limit is assigned with the weight that represents the tolerance of dissimilarity between neighbor, generate the minimum spanning tree (or approximate with it) of this figure, and in succession from minimum spanning tree, remove limit to create the spanning forest that its tree is corresponding with image section, the selective dependency on the limit that wherein, remove from spanning forest is in the morphological characteristic on summit or the limit of this forest.

The concrete shortcoming that the inventor points out is by using the size of the part that above-mentioned prior art Standard Selection limit creates to have larger imbalance.Have been found that in searching the prior art process of MST, from the heaviest many limits of original graph, be removed, but find the heaviest residual limit near the leaf of the tree of being everlasting.The removal that this means heavy limit often causes only having the new part of a pixel.

The present invention is by using the new algorithm of selecting to remove which bar limit in each stage from MST to overcome the limitation of prior art.Thought of the present invention is to use the removal on limit by the tolerance of the possibility of the part of generation fair-sized.

Preferably, for selecting the standard on limit to depend on the tolerance of the distance of the leaf of setting from Dao Gaibian place, limit.Alternatively, this standard can be dependent on two trees that create by removing this limit.In one embodiment, this standard can be dependent on big or small tolerance, for example, by number of vertex, measures size.In another embodiment, this standard can be dependent on poor containing between the summation of primitive definition value in the function of the pixel in the tree on this limit and two trees that create by removal limit of look.

This new algorithm has some very attracting features.Searching MST only needs medium computation complexity, and carry out subsequently cut apart very easy.Be different from some algorithms based on threshold value, accurately the quantity of specified portions.First, this algorithm is perfect classification: be divided into cutting apart to be always included in and being divided in more part of cutting apart of given number part.This algorithm is also for example, for many component images data (image of, being described by R, G, B value).For example, by the image (, RGB image) of more than one component statement, limit power can be between these components definitely or the difference of two squares and, maximum absolute difference or any other suitable tolerance.

The present invention also provides the device that is suitable for carrying out the algorithm of fully describing herein, and it can comprise digital circuit in one embodiment.The present invention can implement in data compression algorithm and device and other image or video processing applications.

Accompanying drawing explanation

Fig. 1 is the example of figure;

Fig. 2 is the expression as the 5x4 image of figure;

Fig. 3 is exemplary little luminance picture;

Fig. 4 is that the figure of the limit weighting of Fig. 3 represents;

Fig. 5 is the minimum spanning tree of Fig. 4;

Fig. 6 be being divided into of Fig. 5 two-part MST cut apart;

Fig. 7 is that tripartite MST of Fig. 5 is cut apart;

Fig. 8 shows the successive stages calculating in isolation;

Fig. 9 is the isolation view of Fig. 5;

Figure 10 shows the product of isolation and limit power;

Figure 11 is used isolation processing that Fig. 5 is divided into two-part cutting apart;

Figure 12 shows summit isolation view;

Figure 13 shows limit isolation view;

Figure 14 shows the product of separation number and the limit power of modification;

Figure 15 is used isolation processing that Fig. 5 is cut apart to tripartite cutting apart;

Figure 16 is exemplary test pattern;

Figure 17 shows and uses the algorithm of described prior art Figure 16 to be divided into the tentative result of cutting apart of 64 parts;

Figure 18 shows and uses the algorithm of described prior art Figure 16 to be divided into the result of cutting apart of 4000 parts;

Figure 19 shows and uses embodiments of the invention Figure 16 to be divided into the result of cutting apart of 64 parts;

Figure 20 shows and simplifies isolation calculating;

Figure 21 shows and uses optional embodiment of the present invention Figure 16 to be divided into the segmentation result of 64 parts;

Figure 22 shows the first stage in the exemplary algorithm of calculating based on energy;

Figure 23 shows the subordinate phase in the algorithm calculating based on energy;

Figure 24 shows the phase III in the algorithm calculating based on energy; And

Figure 25 shows the fourth stage in the algorithm calculating based on energy.

Embodiment

Can be in each stage computed image of cutting apart the new features (being called " isolation (seclusion) " herein) of each pixel.Consider that the leaf of tree is by " exposure ", we conclude that in some sense " far " is " isolation " from the summit of leaf.Thereby isolation is the degree of summit isolation.

Can calculate by following algorithm the isolation of tree or forest:

Current separation number S is made as to O

When existence remains in the limit in forest:

O increases S

O is made as S by the isolation on all leaves summit

O removes all leaves summit and the limit being connected with them

If remain a summit, its isolation be made as to S+1

Fig. 8 shows and how for each pixel in the figure of Fig. 5, to calculate and to isolate.In each stage, removed leaf is coated with white.

We have described the isolation of how to calculate all summits in tree or forest.Can for example, according to the isolation on the isolation on two summit (, the minimum value of two separation number) definition limit.Figure 9 illustrates the isolation view on the limit of Fig. 5.

Alternatively, can use the invulnerable release of above-mentioned isolated algorithm directly to calculate the separation number on limit.

For the limit of selecting to remove, for example, by multiplication, in conjunction with former initial line, weigh and limit separation number.Figure 10 shows the product of former initial line power and isolation view.

For image being divided into 2 parts, can select to have the limit of associated value 18, generate cutting apart shown in Figure 11.In order to continue this cutting procedure, recalculate the isolation view of the forest shown in Figure 11.The summit isolation view generating has been shown in Figure 12, limit isolation view has been shown in Figure 13.Figure 14 shows the product of new separation number and limit power.

Often occur that when using integer to process existence is about the next one uncertain situation of selection of high boundary values.If our balance is conducive to the boundary values arranged side by side of higher isolation, we will select the left side on the limit that two values are 8, and obtain cutting apart shown in Figure 15.

Above example has illustrated that how using isolation view to assist in ensuring that MST is cut apart avoids generating very little part at the commitment of process.Certainly appending to the importance in isolation and cut down between the importance of forest at place, heavy limit, exist compromise.In this compromise selection that can be reflected in the function of weighing in conjunction with isolation and limit.

The tolerance that isolation can be regarded as to " one dimension " because its estimate from limit or summit to the distance of the leaf of tree.The simple modification of isolation processes can be used for measuring the number on summit, that is, and and by removing two trees of limit generation " area ".In this modification, by by the amount being associated with just removed leaf and be connected the amount of the cumulative tolerance in phase Calais, summit of these leaves.

In above-mentioned isolation tolerance or modification tolerance, recursive procedure also can comprise cumulative limit power self, makes directly to calculate weighting separation number.Thereby do not need separation number to be multiplied by the limit of limit power for selecting to remove, and only maximize the tolerance of weighting.

Now the explanation of the advantage of cutting apart the isolation processing in real image will be given in.Figure 16 shows from the luminance component of " form pond (formal pond) " image of EBU regular set that down-converts to the test slide of 360x288 pixel.Figure 17 and Figure 18 show and use respectively above-mentioned prior art MST algorithm to attempt this image to be divided into the resulting segment boundary of 64 and 4,000 part, and Figure 19 shows the result of using the MST algorithm with above-mentioned isolation processing to be divided into 64 parts.

Do not need accurate Calculation isolation view.For example,, because raster scan pattern (pattern) is followed in this processing, so can remove the good approximation that leaf obtains isolation view from forest by " original place ".Figure 20 illustrates the process of this shortening.

This simplification is asymmetric and must is faulty, but on the overall performance of algorithm, affects very little in a particular embodiment.For example, Figure 21 shows this simplification compared with " correctly " isolation processing being used in Figure 19 to " form pond " being separated into the impact in 64 regions.

Similarly cylinderization can be used for the revision of previously described isolation tolerance.

Can also reduce the processing time by searching the approximate of MST.For example,, during Prim algorithm operating, for example, if according to some standards (, distribution based on precalculated limit power), think its weight " enough little ", can add limit, therefore need to be in whole tree of each stage Search not search the minimal weight of adjacent side.

To describe now the present invention further improves.In the given stage of cutting procedure, what weighed on separation number and limit is used in combination be used to definite which bar limit of removing from forest.This decision can be thought and comprises two parts: determine in which tree from forest and remove limit, and determine from tree to remove which bar limit.In a preferred embodiment of the invention, other of this tree easily mensurable characteristic can be used for forming the first determining, then the present invention's algorithm itself can be used for forming the second portion determining.For example, can select to there is the highest tree total or average limit power, or employing have the tree on maximum summits or the combination of some other tolerance or tolerance.This embodiment has two advantages.First advantage is can consider additional information when determining separated which tree.Second advantage is only need to calculate separation number for selected tree.

Can find out, conventionally partly by above-mentioned calculating isolation processes, obtain the simple tree only with two leaves.The residue process of said process is only in succession removed these two leaves and is increased current separation number.On the contrary, in one embodiment, when acquisition has the simple tree of two leaves, stop this process, and current separation number is distributed to the remainder of tree.This modification can be accelerated isolation and calculate, or at least avoids getting back to the second algorithm for the treatment of two leaf trees.In some cases, this optional algorithm can cause the improvement of subjective performance, because it has limited with the limit power in tree central area, compares and seems to isolating excessive emphasizing.

In another embodiment, can during the removal on tree limit, calculate second or optional amount or tolerance.This second or optional amount or tolerance be the amount of some functions of the tree that reduces of the removal on limit.

The function of if tree T is defined as f (T), and the removal of limit e is divided into two trees U and V by tree T, and the amount of the function that removal on limit reduces can be provided by following formula:

f{T)-f{U)-f{V}

Suitable function can represent " energy " of tree.The example of this energy function is

$E_T=\underset{i\∈T}{\mathrm{\Σ}}{(x_i-\stackrel{\‾}{x})}^2$

Wherein

X _ibrightness value for pixel I place;

T is considered tree; And

mean value for brightness in tree.

Use above definition, according to the energy minimizing (it can regard " energy of limit e " as) of above formula, can be written as

$E_e=f\left(T\right)-f\left(U\right)-f\left(V\right)=E_T-E_U-E_V$

$=\underset{i\∈T}{\mathrm{\Σ}}{(x_i-{\stackrel{\‾}{x}}_T)}^2-\underset{i\∈U}{\mathrm{\Σ}}{(x_i-{\stackrel{\‾}{x}}_U)}^2-\underset{i\∈V}{\mathrm{\Σ}}{(x_i-{\stackrel{\‾}{x}}_V)}^2$

$=\underset{i\∈U}{\mathrm{\Σ}}{(x_i-{\stackrel{\‾}{x}}_T)}^2+\underset{i\∈V}{\mathrm{\Σ}}{(x_i-{\stackrel{\‾}{x}}_T)}^2-\underset{i\∈U}{\mathrm{\Σ}}{(x_i-{\stackrel{\‾}{x}}_U)}^2-\underset{i\∈V}{\mathrm{\Σ}}{(x_i-{\stackrel{\‾}{x}}_V)}^2$

$=\underset{i\∈U}{\mathrm{\Σ}}{(x_i-{\stackrel{\‾}{x}}_U+{\stackrel{\‾}{x}}_U-{\stackrel{\‾}{x}}_T)}^2+\underset{i\∈V}{\mathrm{\Σ}}{(x_i-{\stackrel{\‾}{x}}_V+{\stackrel{\‾}{x}}_V-{\stackrel{\‾}{x}}_T)}^2-\underset{i\∈U}{\mathrm{\Σ}}{(x_i-{\stackrel{\‾}{x}}_U)}^2-\underset{i\∈V}{\mathrm{\Σ}}{(x_i-{\stackrel{\‾}{x}}_V)}^2$

$=N_U{({\stackrel{\‾}{x}}_U-{\stackrel{\‾}{x}}_T)}^2+N_V{({\stackrel{\‾}{x}}_V-{\stackrel{\‾}{x}}_T)}^2$

$=N_U{({\stackrel{\‾}{x}}_U-{\stackrel{\‾}{x}}_T)}^2(1+\frac{N_U}{N_T-N_U})$

Wherein, N refer to shown in the quantity on summit in tree.

The algorithm that calculates isolation can easily be suitable for calculating this amount on tree.This can be by counting the summit quantity running into and determining that the summation of the pixel value running into completes when starting inwardly to process from leaf.Can explain in more detail this algorithm with reference to Figure 22 to Figure 25.The identical input data of Figure 22 to Figure 25 based on used in previous description.

In these figure, pixel value (being brightness value in this case) is illustrated in the circle corresponding to the summit of each pixel or figure.The limit of minimum spanning tree is shown the line of connect Vertex.

In this exemplary embodiment, in the row from left to right of figure and from top to bottom, with raster order considered pixel.Therefore,, with raster order scanning, identified and processed the leaf (being only connected to summit on one side) of tree.At every one-phase, the cumulative sum S of the number of vertex n that record runs into and the brightness value on these summits.At the leaf place of tree, n=1 and s are pixel value.The mode that processed summit has been surrounded by square frame in Figure 22 illustrates.

Once summit is processed, just from tree, remove corresponding limit.Therefore, at leaf newly-generated during the process of raster, can in identical raster process, run into after a while, and also can be processed as the situation at the place, upper left at figure.Therefore, this raster process generates the calculating of the summation of 8 in 20 summits of cumulative as shown in figure 22 number of vertex and figure first stage.

In Figure 23, the limit of removing during the first stage is now shown in broken lines.Show new leaf, and due to raster scan pattern, can in a process, again process several summits.

In Figure 24, only two leaves of residue tree and these are all removed.Remain a pixel (thering is value 8) for isolation separately, and owing to having removed all limits from figure, so do not need to process this pixel.Note, do not guarantee that three processes are enough to remove all limits in the figure of this size.

The last stage is the accumulated value of the pixel of the processing based on being responsible for removing on this limit, returns through all limits and the formula applying top and derive obtains the energy value on every limit.In the situation that the average brightness of given tree gulps down, be 4.65, the accumulative total of the pixel value of all 20 pixels in example images is 93.Therefore, for example, the following limit energy that calculates the 3rd limit starting from the left side of the end row of Figure 24:

N _U＝10

${\stackrel{\‾}{x}}_U=s/N_U=30/10=3$

${\stackrel{\‾}{x}}_T=4.65$

According to the limit energy of above formula 54.5.

Figure 25 shows each energy value on all limits.

When use this algorithm to select will to remove limit time, the limit of selecting energy to be maximized.In this example, this is the limit with energy 57.4, and this edge will be removed.

Energy metric also can be weighed with limit, isolate or above-mentioned other amount combination, to obtain the more complicated standard of selecting for limit.

With reference to two dimensional image, described according to partitioning algorithm of the present invention.By the internuncial suitable definition between the pixel in consecutive image, identical algorithm can be used for image sequence.

Claims (10)

1. an image segmentation algorithm, wherein, pixel with image described in the vertex representation of figure, the adjacency that represents pixel with the limit of described figure, for described limit, distribute the weight of the tolerance that represents the dissimilarity between neighbor, generate the minimum spanning tree (or approximate with it) of described figure, and in succession from described minimum spanning tree, remove limit to create spanning forest, the tree of described forest is corresponding to the part of described image

Wherein, the limit that selection will be removed from described spanning forest depends on the morphological characteristic to the described summit of described forest or the calculating of described limit.

2. algorithm according to claim 1, wherein, for selecting the tolerance on limit to depend on from described limit the tolerance of distance of leaf of the tree at place, described limit.

3. algorithm according to claim 1, wherein, for selecting the tolerance on limit to depend on by removing the tolerance of two trees that described limit creates.

4. algorithm according to claim 3, wherein, the described tolerance on limit is the function of the described pixel in the described tree that comprises described limit and by removing poor between the summation of value of described function of the described pixel in described two trees that described limit creates.

5. algorithm according to claim 4, wherein, described function is energy function.

6. algorithm according to claim 5, wherein, in tree, the described pixel value of the energy value of pixel and pixel in described tree and the difference between the average pixel value of described tree is square relevant.

7. according to algorithm in any one of the preceding claims wherein, wherein, by recursive algorithm, calculate the described tolerance on every limit in tree.

8. algorithm according to claim 7, wherein, described tolerance is configured to connect the initial value on first group of limit of the leaf of described tree, wherein, for the follow-up one group of limit that connects described first group of limit, increase or cumulative described tolerance, and wherein, repeat described processing until all limits are all taken into account.

9. according to the algorithm described in claim 7 or 8, wherein, described tolerance is configured to connect the initial value on limit of the leaf of described tree, removes those limits temporarily, increases or cumulative described tolerance also repeats described process until there is no remaining sides.

10. an image segmentation algorithm, wherein, pixel with image described in the vertex representation of figure, the adjacency that represents pixel with the limit of described figure, for described limit, distribute the weight of the tolerance that represents the dissimilarity between neighbor, generate the minimum spanning tree (or approximate with it) of described figure, and in succession from described minimum spanning tree, remove limit to create spanning forest, the tree of described forest is corresponding to the part of described image

Wherein, select to depend on from the limit of described spanning forest removal the tolerance of two trees that create by removal limit.

Images