CN106981068B - A kind of interactive image segmentation method of joint pixel pait and super-pixel - Google Patents
A kind of interactive image segmentation method of joint pixel pait and super-pixel Download PDFInfo
- Publication number
- CN106981068B CN106981068B CN201710217696.7A CN201710217696A CN106981068B CN 106981068 B CN106981068 B CN 106981068B CN 201710217696 A CN201710217696 A CN 201710217696A CN 106981068 B CN106981068 B CN 106981068B
- Authority
- CN
- China
- Prior art keywords
- pixel
- super
- node
- background
- segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 49
- 238000003709 image segmentation Methods 0.000 title claims abstract description 16
- 230000011218 segmentation Effects 0.000 claims abstract description 56
- 230000000877 morphologic effect Effects 0.000 claims abstract description 12
- 238000004040 coloring Methods 0.000 claims description 11
- 239000011159 matrix material Substances 0.000 claims description 8
- 238000000513 principal component analysis Methods 0.000 claims description 4
- 230000010339 dilation Effects 0.000 claims description 3
- 230000003628 erosive effect Effects 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 230000003993 interaction Effects 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 241000282472 Canis lupus familiaris Species 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 239000000284 extract Substances 0.000 description 3
- 230000036961 partial effect Effects 0.000 description 3
- 238000000354 decomposition reaction Methods 0.000 description 2
- 235000013399 edible fruits Nutrition 0.000 description 2
- 230000002829 reductive effect Effects 0.000 description 2
- 238000000638 solvent extraction Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000013316 zoning Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
Landscapes
- Image Analysis (AREA)
Abstract
The present invention provides the interactive image segmentation method of a kind of joint pixel pait and super-pixel, pre-segmentation is carried out to input picture including step S1, using MeanShift algorithm, super-pixel is extracted, and color histogram is used to be modeled the distribution of color to indicate super-pixel to each super-pixel；S2, the interactive information inputted according to user, establish graph model by node of super-pixel, complete the segmentation of super-pixel grade using max-flow min-cut algorithm；S3, on the basis of super-pixel grade segmentation result, using morphological operation object edge construct a narrowband region；It S4, is that corresponding foreground and background model is established in foreground and background region respectively, and establishes graph model by node of pixel to narrowband region, then complete Pixel-level segmentation using max-flow min-cut algorithm on the narrowband constructed.The present invention improves execution efficiency by organically combining Pixel Information and super-pixel information, and obtains more accurate segmentation result under the conditions of user's interaction few as far as possible.
Description
Technical field
The present invention relates to technical field of image processing, and in particular to a kind of interactive image point of joint pixel pait and super-pixel
Segmentation method.
Background technique
Image segmentation is a basic problem in image procossing, and main purpose is by interesting target from complex background
In extract, to carry out target detection, tracking, identification and scene analysis etc., in pattern-recognition and computer vision etc.
There is very extensive application in field.Whether priori knowledge is provided according to user, the method that can divide the image into is divided into automatic point
Segmentation method and interactive segmentation method.For the natural image of those types multiplicity, content complexity, interactive segmentation method makes
Segmentation result more meets and can directly react the subjective desire of user, therefore is gradually taken seriously.Wherein, GraphCut
Interactive segmentation method because of it with global optimum, numerical value strong robustness, execution efficiency be high, topological structure of segmentation weighted graph from
By and N-D image segmentation ability etc., and obtain the extensive concern of researchers.
One outstanding interactive image segmentation system must satisfy the feedback time that user few as far as possible is interactive, short as far as possible
And the conditions such as segmentation result as accurate as possible.GraphCut algorithm is oriented using pixel as node one weighting of building
Then figure carries out global solution using max-flow min-cut algorithm.Regrettably, the algorithm is merely with image grayscale information, and
Colouring information is not considered, so that segmentation result is unsatisfactory when picture material is more complex；However when foreground and background is in ash
When spending very close in information, a large amount of interaction of user's input is usually required to obtain ideal segmentation result GraphCut algorithm
Information；Further, since be using pixel as node, with the increase of picture size, the node in graph model that thus constructs
Number significantly increases, and takes considerable time so as to cause the execution needs of max-flow min-cut algorithm.For these disadvantages, Li
Et al. propose Lazy Snapping partitioning algorithm.The algorithm uses watershed (WaterShed) algorithm to carry out image first
Cut zone is rebuild a weighted digraph by pre-segmentation, is then carried out using max-flow min-cut algorithm
The overall situation solves, and finally makes segmentation result more accurate using a set of scheme that manually adjusts again.Only drawback is that the algorithm
There are several main problems: since WaterShed algorithm is merely with the gradient information of gray level image, over-segmentation phenomenon is serious, and
And the number of regions in its pre-segmentation result is still more；The algorithm characterizes each region, this representation using color mean value
It is too simple, it can not accurately indicate the distribution of color in each region；Select relatively simple K-Means poly- in modeling process
Class algorithm, Clustering Effect are affected by primary condition and disturbing factor；The algorithm needs a large amount of thick in interactive process
The fine tuning that reconciles operates, so that entire interactive process is excessively cumbersome etc..
For opposite WaterShed algorithm, MeanShift partitioning algorithm has obtained more by its outstanding segmentation performance
Widely to study and applying.The algorithm takes full advantage of colouring information, and over-segmentation phenomenon is lighter, and cut zone number obtains
To significantly reducing.Ning et al. carries out pre-segmentation using MeanShift algorithm, and indicates each area using color histogram
Domain proposes a kind of region merging technique (MSRM) algorithm based on maximum similarity.Different from max-flow min-cut algorithm, MSRM is calculated
Method merges mechanism using a kind of region automatically and completes color images.But the present inventor has found after study, should
Algorithm only considers interregional similarity, and does not consider the correlation between each region and interactive information, and in region merging technique mistake
The similarity between multiple statistical regions color histogram and multiple zoning is needed in journey so that entire algorithm when be spaced apart
Sell larger；In addition, MSRM algorithm does not account for the over-segmentation problem of MeanShift algorithm, therefore there is big in object edge
Amount is accidentally divided.
For simplified user interactive process, Rother et al. proposes GrabCut algorithm.The algorithm is according to the square of user's mark
Initial foreground and background model is established in shape region, due to simultaneously including foreground information and background information inside rectangular area,
The algorithm constantly updates foreground and background model iteration in a manner of learning, until the energy convergence that global solution obtains
When iteration stopping.But the present inventor has found after study, since GrabCut algorithm only determines background area, and does not have
Have and accurately determine foreground area, therefore initial back-ground model establishes accurate whether will directly affect final segmentation result.
Therefore, for some problems existing for preceding method, a kind of efficiently and accurately interactive image segmentation side is designed
Method is particularly important.
Summary of the invention
For technical problem present in conventional images dividing method, the present invention provides a kind of joint pixel pait and super-pixel
Interactive image segmentation method, this method is by organically combining Pixel Information and super-pixel information, to improve existing interactive mode
The execution efficiency of image partition method, and more accurate segmentation result is obtained under the conditions of user's interaction few as far as possible.
In order to solve the above-mentioned technical problem, present invention employs the following technical solutions:
A kind of interactive image segmentation method of joint pixel pait and super-pixel, method includes the following steps:
S1, pre-segmentation is carried out to input picture using MeanShift algorithm, to complete super-pixel extraction, and uses color
Histogram models each super-pixel, accurately to indicate the distribution of color of super-pixel；
S2, the interactive information inputted according to user establish graph model by node of super-pixel, and use max-flow min-cut
Algorithm carries out global solution, to complete super-pixel grade cutting procedure；
S3, on the basis of super-pixel grade segmentation result, using morphological operation object edge construct narrowband area
Domain；
S4, corresponding foreground and background model is established respectively on the narrowband constructed for foreground and background region, and right
Narrowband region establishes graph model by node of pixel, finally carries out global solution using max-flow min-cut algorithm again, with
Complete Pixel-level cutting procedure.
Further, the step S2 the following steps are included:
S21, interactive information is inputted by user, F is denoted as labeled as the super-pixel set of target, labeled as the super-pixel of background
Set is denoted as B, and unmarked super-pixel set is denoted as U；
S22, pre-segmentation image obtained in step S1 is expressed as a direct graph with weight G=by node of super-pixel
(V, E, W), wherein V indicates the set of figure interior joint, each super-pixel in correspondence image；E indicates the set on side in figure, corresponding
The side between neighbouring super pixels is connected in image；W indicates the weight on side in figure, for indicating that adjacent node is divided into same generic
Tendency degree；For any two neighbouring super pixels i and j, using its color histogram, its similarity is calculated using Pasteur's distance
δ (i, j), and the weight for indicating the two neighbouring super pixels side；
S23, foreground and background is modeled respectively using color histogram according to the interactive information of user's input, and structure
Make two dummy nodes: the sink node of the source node of corresponding foreground model and corresponding background model；Wherein, in set V
Arbitrary node and the weight on the formed side of source and sink node respectively indicate the node and be under the jurisdiction of inclining for foreground and background
To degree；For unmarked super-pixel i, using its color histogram, itself and source node are calculated separately using Pasteur's distance
With the similarity δ (i, F) and δ (i, B) of sink node；
S24, it is directed to above-mentioned graph model, is defined as follows energy function:
Wherein, first item is area information item, for measuring the similarity degree between each super-pixel and interactive information；Section 2
For marginal information item, for measuring the similarity degree between super-pixel；The definition difference of area information item and marginal information item is as follows:
b(ιi,ιj)=λ δ (i, j) formula (7)
Wherein, ιiIndicate the generic number of node i, ιi=1 corresponds to prospect, ιi=0 corresponds to background, Indicate the Neighbourhood set of super-pixel node, λ is control parameter；
On the basis of good graph model established above, executing max-flow min-cut algorithm be can be obtained based on super-pixel grade
Globally optimal solution.
Further, the step S3 the following steps are included:
S31, note are S by the bianry image that step S2 is divided, by carrying out morphological erosion behaviour to bianry image S
Make, can be obtained and contain only mesh target area SF=S Θ b, respective pixel set are denoted as TrimapForeground；Wherein b is (2d
+ 1) × (2d+1) the square structure element of size, d is positive integer in formula；
S32, to bianry image S carry out morphological dilation, and with target area SFDoing difference operation can be obtained one
Simultaneously containing the narrowband region of target and backgroundRespective pixel set is denoted as
TrimapUnknown；
S33, the region for containing only background information are represented by SB=S- (SU+SF), respective pixel set is denoted as
TrimapBackground can be obtained by a ternary mask images in this way, including well-established narrowband region SU。
Further, the step S4 the following steps are included:
S41, by target area S obtained in step S3FWith background area SBRegard virtual source node and virtual as respectively
Then sink node establishes prospect mixed Gauss model and background mixed Gauss model using Principal Component Analysis to it respectively,
Gauss number is set as K；
S42, graph model is established to narrowband region obtained in step S3 using pixel as node, and to formula (5) energy
Area information item and marginal information item in function are redefined as follows:
Wherein, dist (i, j) indicates the space length of pixel i and j, IiIndicate the colouring information of pixel i, IjIt indicates
The colouring information of pixel j,8 neighborhoods are used during establishing graph model, because of λ
≥b(ιi,ιj), so the desirable λ+1 of κ=8；Parameter σ is used to control colouring information difference degree, Dx(i) ith pixel point is indicated
The distribution situation in prospect or background model, wherein x ∈ { F, B }, value can be calculated from following formula:
Wherein,Indicate the weight shared by k-th of Gaussian Profile in the mixed Gauss model of prospect or background,WithThe mean value and covariance matrix of k-th of Gaussian Profile in prospect or background model are respectively indicated,Indicate covariance square
Battle arrayDeterminant；
On the basis of good graph model established above, executing max-flow min-cut algorithm be can be obtained based on the complete of Pixel-level
Office's optimal solution.
Compared with prior art, joint pixel pait provided by the invention and the interactive image segmentation method of super-pixel have with
Lower advantage:
The first, the present invention carries out pre-segmentation using MeanShift algorithm, its interstitial content of the digraph thus constructed obtains
It is greatly reduced, to effectively increase the execution efficiency of max-flow min-cut algorithm；For each super-pixel, use more
Effective color histogram representation, and foreground and background model equally uses color histogram to indicate, and no longer need
Carry out additional modeling process；In addition, tutorial message is provided since super-pixel interacts for user, so that the present invention only needs less
Simultaneously more satisfactory segmentation effect can be obtained in the interactive information of amount within a short period of time.
The second, during establishing graph model, the present invention has considered not only interregional similarity degree, but also examines
The similarity degree between each region and foreground and background is considered, due to taking full advantage of interactive information, so that segmentation result is more
It is accurate；In addition, the similarity between each adjacent area only needs to calculate once, then transfer at max-flow min-cut algorithm
Reason, to obtain the segmentation of super-pixel grade as a result, so that the space-time expense of this method has obtained further being greatly lowered.
Third, because inevitably there is boundary leakage in cutting procedure in MeanShift algorithm, lead to partial segmentation area
Foreground and background two parts are crossed over simultaneously so that occurring accidentally dividing in domain, and the present invention, which is divided using morphological operation in super-pixel grade, to be tied
Fruit marginal portion constructs a narrowband region, then establishes graph model as node using pixel to the narrowband region and executes max-flow
Minimal cut algorithm, to effectively increase marginal portion segmentation precision.
Detailed description of the invention
Fig. 1 is the interactive image segmentation method flow diagram of joint pixel pait provided by the invention and super-pixel.
Fig. 2 is the schematic diagram provided by the invention that narrowband region is established on the basis of super-pixel grade segmentation result.
Fig. 3 is provided by the invention to Flower image interactive information schematic diagram input by user.
Fig. 4 be it is provided by the invention to Flower image be respectively adopted GraphCut, Lazy Snapping, GrabCut,
MSRM and the obtained segmentation effect figure of the method provided by the present invention.
Fig. 5 is provided by the invention to Dogs image interactive information schematic diagram input by user.
Fig. 6 be it is provided by the invention to Dogs image be respectively adopted GraphCut, Lazy Snapping, GrabCut,
MSRM and the obtained segmentation effect figure of the method provided by the present invention.
Specific embodiment
In order to be easy to understand the technical means, the creative features, the aims and the efficiencies achieved by the present invention, tie below
Conjunction is specifically illustrating, and the present invention is further explained.
It please refers to shown in Fig. 1, the present invention provides the interactive image segmentation method of a kind of joint pixel pait and super-pixel, the party
Method the following steps are included:
S1, pre-segmentation is carried out to input picture using MeanShift algorithm, to complete super-pixel extraction, and uses color
Histogram models each super-pixel, accurately to indicate the distribution of color of super-pixel.As specific embodiment, input one
Width color image carries out pre-segmentation to it using MeanShift algorithm to obtain super-pixel；Due to the region of each super-pixel
Area is relatively large, therefore the present invention models each super-pixel using color histogram, accurately to indicate super-pixel
Distribution of color situation.If color histogram size is n3, for any pixel p=(x, y) in color image I, pixel
Value is denoted as I (p)=(r, g, b), and new color value indicates after quantifying to it are as follows:
I ' (p)=(r/l) n2+ (g/l) n+ (b/l) formula (1)
Wherein ,/it is floor operation,N takes 16 to can be obtained preferable result in the present embodiment.
Then count what color value k=I ' (p) after all pixels point quantifies in each super-pixel (assuming that number is i) occurred
Number Li(k), wherein k ∈ [0, n3), and final color histogram H can be obtained after normalizing to iti, corresponding color value k
Corresponding histogram value Hi(k) it calculates as follows:
Wherein NiFor the sum of all pixels in i-th of super-pixel, i.e.,
S2, the interactive information inputted according to user establish graph model by node of super-pixel, and use max-flow min-cut
Algorithm carries out global solution, to complete super-pixel grade cutting procedure.As specific embodiment, the step S2 includes following step
It is rapid:
S21, interactive information is inputted by user, F is denoted as labeled as the super-pixel set of target, labeled as the super-pixel of background
Set is denoted as B, and unmarked super-pixel set is denoted as U；
S22, pre-segmentation image obtained in step S1 is expressed as a direct graph with weight G=by node of super-pixel
(V, E, W), wherein V indicates the set of figure interior joint, each super-pixel in correspondence image；E indicates the set on side in figure, corresponding
The side between neighbouring super pixels is connected in image；W indicates the weight on side in figure, for indicating that adjacent node is divided into same generic
Tendency degree；
For arbitrary neighborhood super-pixel i and super-pixel j, color histogram is respectively HiAnd Hj, the present invention using Pasteur away from
From its similarity of calculating, and it is used to indicate the weight on the two neighbouring super pixels side:
Wherein δ (i, j) is bigger, indicates that the color histogram of super-pixel i and super-pixel j is more close, corresponds to two regions also just
It is more similar, then super-pixel i and super-pixel j belong to same generic probability it is also bigger, otherwise super-pixel i and super-pixel j belong to
The probability of same generic is also just smaller.
S23, foreground and background is modeled respectively using color histogram according to the interactive information of user's input, and structure
Make two dummy nodes: the sink node of the source node of corresponding foreground model and corresponding background model；Wherein, in set V
Arbitrary node and the weight on the formed side of source and sink node respectively indicate the node and be under the jurisdiction of inclining for foreground and background
To degree.It is right respectively using the color histogram in step S1 for interactive information, that is, target F and background B that user provides
It establishes target and background model, is denoted as HFAnd HB。
For unmarked super-pixel i, using its color histogram, it is calculated separately with source node and sink node
Similarity equally calculates separately the similarity with target F and background B using Pasteur's distance here:
S24, the present invention not only consider the similitude between super-pixel, but also also contemplate the pass of each super-pixel and interactive information
System, therefore it is directed to above-mentioned graph model, it is defined as follows energy function:
Wherein, first item is area information item, for measuring the similarity degree between each super-pixel and interactive information；Section 2
For marginal information item, for measuring the similarity degree between super-pixel；The definition difference of area information item and marginal information item is as follows:
b(ιi,ιj)=λ δ (i, j) formula (7)
Wherein, ιiIndicate the generic number of node i, ιi=1 corresponds to prospect, ιi=0 corresponds to background, Indicate the Neighbourhood set of super-pixel node, λ is control parameter；
On the basis of good graph model established above, executing max-flow min-cut algorithm be can be obtained based on super-pixel grade
Globally optimal solution.
S3, it obtains in MeanShift algorithm since there is certain over-segmentation phenomenons in super-pixel, so that aforementioned super
Object edge part in pixel segmentation result is simultaneously rough, and there is the segmentations of certain mistake, therefore the present invention is in super-pixel grade
On the basis of segmentation result, a narrowband region is constructed in object edge using morphological operation, to object edge partial region
Secondary splitting is carried out, to further increase segmentation precision.As specific embodiment, the step S3 the following steps are included:
S31, note are S by the bianry image that step S2 is divided, as shown in Fig. 2 (a).In order to establish one in object edge
A narrowband region, the present invention carry out morphological erosion operation to bianry image S first, can be obtained and contain only mesh target area SF=
S Θ b, respective pixel set are denoted as TrimapForeground, such as the white area in Fig. 2 (b)；Wherein b is (2d+1) × (2d
+ 1) the square structure element of size, d is positive integer, d ∈ [2,5] in embodiment in formula；
S32, then to bianry image S carry out morphological dilation, and with target area SFDoing difference operation can be obtained
One simultaneously containing the narrowband region of target and backgroundRespective pixel set is denoted as
TrimapUnknown, such as the gray area in Fig. 2 (b)；
S33 and the region for containing only background information is represented by SB=S- (SU+SF), respective pixel set is denoted as
TrimapBackground can be obtained by a ternary mask images such as the black region in Fig. 2 (b) in this way
(TrimapForeground, TrimapBackground, TrimapUnknown), including well-established narrowband region
SU。
S4, corresponding foreground and background model is established respectively on the narrowband constructed for foreground and background region, and right
Narrowband region establishes graph model by node of pixel, finally carries out global solution using max-flow min-cut algorithm again, with
Complete Pixel-level cutting procedure.As specific embodiment, the step S4 the following steps are included:
S41, by target area S obtained in step S3FWith background area SBRegard virtual source node and virtual as respectively
Then sink node establishes prospect mixed Gauss model and background mixed Gaussian using principal component analysis (PCA) method to it respectively
Model, Gauss number are set as K, K=5 in embodiment；The tool of the prospect mixed Gauss model and background mixed Gauss model
Body method for building up is as follows:
S411, according to the pixel color information of prospect F or background B, calculate its covariance matrix ∑FOr ∑B, and to its into
Row feature decomposition extracts principal component or principal direction, i.e. feature vector corresponding to maximum eigenvalue, and sample data is in the principal direction
Upper projection value has maximum variance, then utilizes the projection result of sample data in a main direction by entire distribution space average mark
At two sub-spaces；
S412, selection has the subspace of maximal projection variance distribution in all subspaces, calculates institute in the subspace
There is the covariance matrix of sample, feature decomposition is carried out to it, extracts principal direction, and this subspace is divided into two in the direction
A new subspace；This step S412 is repeated, until obtaining K sub-spaces；
S413, the mean value of sample data in K sub-spaces, covariance matrix and the shared power of distribution are finally calculated separately
Weight, can finally obtain prospect or the respective mixed Gauss model of background.
S42, due to narrowband region SUThe usual very little of area, therefore its contained pixel number is seldom, calculates to solve MeanShift
Method bring over-segmentation problem and the segmentation accuracy for further effectively improving narrowband region, the present invention is using pixel as node
Graph model only is established to narrowband region obtained in step S3, max-flow min-cut algorithm is executed on the basis of this graph model
Obtain the globally optimal solution based on Pixel-level, i.e., final Pixel-level segmentation result.Different from being based on super-pixel grade in step S2
Graph model foundation, for this graph model based on Pixel-level establish in two energy, that is, area information item and marginal information
Item has carried out following redesign, i.e., in formula (5) energy function area information item and marginal information item carry out it is following again
Definition:
Wherein, dist (i, j) indicates the space length of pixel i and j, IiIndicate the colouring information of pixel i, IjIt indicates
The colouring information of pixel j,8 neighborhoods are used during establishing graph model, because of λ
≥b(ιi,ιj), so the desirable λ+1 of κ=8；Parameter σ is used to control colouring information difference degree, Dx(i) ith pixel point is indicated
The distribution situation in prospect or background model, wherein x ∈ { F, B }, value can be calculated from following formula:
Wherein,Indicate the weight shared by k-th of Gaussian Profile in the mixed Gauss model of prospect or background,With
The mean value and covariance matrix of k-th of Gaussian Profile in prospect or background model are respectively indicated,Indicate covariance matrixDeterminant.
It please refers to shown in Fig. 3, Fig. 4, Fig. 5 and Fig. 6, specially Flower image and Dogs image is respectively adopted
The segmentation result comparison of GraphCut, Lazy Snapping, GrabCut, MSRM and the method provided by the present invention.For Flower
Image, to even things up, GraphCut, Lazy Snapping, MSRM and the method provided by the present invention have used identical interaction letter
Breath, it is specific as shown in Fig. 3 (a), and shown in such as Fig. 3 (b) of interactive information used in GrabCut algorithm.For Dogs image,
GraphCut, Lazy Snapping and tri- kinds of methods of MSRM have used identical interactive information, specific as shown in Fig. 5 (a),
Shown in such as Fig. 5 (b) of interactive information used in GrabCut algorithm, since super-pixel provides tutorial message for user, this
Invention providing method only needs less interactive information, as shown in Fig. 5 (c).It can from the segmentation result comparison in Fig. 4 and Fig. 6
Out, extracted target information is not in the segmentation result obtained by GraphCut, Lazy Snapping and tri- kinds of methods of MSRM
Completely, specific such as Fig. 4 and (a), (b) and (c) in Fig. 6 are shown；And extracted target in the result obtained by MSRM method
But there is accidentally dividing, (d) in specific such as Fig. 4 and Fig. 6 is shown at edge；In comparison, it is obtained by the method provided by the present invention
Result in extracted Target Segmentation more complete and accurate, shown in specific (e) as in Fig. 4 and Fig. 6.
Compared with prior art, joint pixel pait provided by the invention and the interactive image segmentation method of super-pixel have with
Lower advantage:
The first, the present invention carries out pre-segmentation using MeanShift algorithm, its interstitial content of the digraph thus constructed obtains
It is greatly reduced, to effectively increase the execution efficiency of max-flow min-cut algorithm；For each super-pixel, use more
Effective color histogram representation, and foreground and background model equally uses color histogram to indicate, and no longer need
Carry out additional modeling process；In addition, tutorial message is provided since super-pixel interacts for user, so that the present invention only needs less
Simultaneously more satisfactory segmentation effect can be obtained in the interactive information of amount within a short period of time.
The second, during establishing graph model, the present invention has considered not only interregional similarity degree, but also examines
The similarity degree between each region and foreground and background is considered, due to taking full advantage of interactive information, so that segmentation result is more
It is accurate；In addition, the similarity between each adjacent area only needs to calculate once, then transfer at max-flow min-cut algorithm
Reason, to obtain the segmentation of super-pixel grade as a result, so that the space-time expense of this method has obtained further being greatly lowered.
Third, because inevitably there is boundary leakage in cutting procedure in MeanShift algorithm, lead to partial segmentation area
Foreground and background two parts are crossed over simultaneously so that occurring accidentally dividing in domain, and the present invention, which is divided using morphological operation in super-pixel grade, to be tied
Fruit marginal portion constructs a narrowband region, then establishes graph model as node using pixel to the narrowband region and executes max-flow
Minimal cut algorithm, to effectively increase marginal portion segmentation precision.
Finally, it is stated that the above examples are only used to illustrate the technical scheme of the present invention and are not limiting, although referring to compared with
Good embodiment describes the invention in detail, those skilled in the art should understand that, it can be to skill of the invention
Art scheme is modified or replaced equivalently, and without departing from the objective and range of technical solution of the present invention, should all be covered at this
In the scope of the claims of invention.
Claims (3)
1. the interactive image segmentation method of a kind of joint pixel pait and super-pixel, which is characterized in that method includes the following steps:
S1, pre-segmentation is carried out to input picture using MeanShift algorithm, to complete super-pixel extraction, and uses color histogram
Figure models each super-pixel, accurately to indicate the distribution of color of super-pixel；
S2, the interactive information inputted according to user establish graph model by node of super-pixel, and use max-flow min-cut algorithm
Global solution is carried out, to complete super-pixel grade cutting procedure；
S3, on the basis of super-pixel grade segmentation result, using morphological operation object edge construct a narrowband region；
S4, corresponding foreground and background model is established respectively on the narrowband constructed for foreground and background region, and to narrowband
Graph model is established by node of pixel in region, finally carries out global solution using max-flow min-cut algorithm again, to complete
Pixel-level cutting procedure；
Wherein, the step S2 the following steps are included:
S21, interactive information is inputted by user, F is denoted as labeled as the super-pixel set of target, labeled as the super-pixel set of background
It is denoted as B, unmarked super-pixel set is denoted as U；
S22, by pre-segmentation image obtained in step S1 using super-pixel as node be expressed as a direct graph with weight G=(V, E,
W), wherein V indicates the set of figure interior joint, each super-pixel in correspondence image；E indicates the set on side in figure, correspondence image
Side between middle connection neighbouring super pixels；W indicates the weight on side in figure, for indicating that adjacent node is divided into inclining for same generic
To degree；For any two neighbouring super pixels i and j, using its color histogram, using Pasteur's distance calculate its similarity δ (i,
J), and for indicating the weight on the two neighbouring super pixels side；
S23, foreground and background is modeled respectively using color histogram according to the interactive information of user's input, and constructs two
A dummy node: the sink node of the source node of corresponding foreground model and corresponding background model；Wherein, appointing in set V
The weight on meaning node and the formed side of source and sink node respectively indicates the tendency journey that the node is under the jurisdiction of foreground and background
Degree；For unmarked super-pixel i, using its color histogram, using Pasteur's distance calculate separately its with source node and
The similarity δ (i, F) and δ (i, B) of sink node；
S24, it is directed to above-mentioned graph model, is defined as follows energy function:
Wherein, first item is area information item, for measuring the similarity degree between each super-pixel and interactive information；Section 2 is side
Edge item of information, for measuring the similarity degree between super-pixel；The definition difference of area information item and marginal information item is as follows:
b(ιi,ιj)=λ δ (i, j) formula (7)
Wherein, ιiIndicate the generic number of node i, ιi=1 corresponds to prospect, ιi=0 corresponds to background, Indicate the Neighbourhood set of super-pixel node, λ is control parameter；
On the basis of good graph model established above, executing max-flow min-cut algorithm can be obtained the overall situation based on super-pixel grade
Optimal solution.
2. the interactive image segmentation method of joint pixel pait according to claim 1 and super-pixel, which is characterized in that described
Step S3 the following steps are included:
S31, note are S by the bianry image that step S2 is divided, by carrying out morphological erosion operation to bianry image S, i.e.,
It can obtain containing only mesh target area SF=S Θ b, respective pixel set are denoted as TrimapForeground；Wherein b be (2d+1) ×
The square structure element of (2d+1) size, d is positive integer in formula；
S32, to bianry image S carry out morphological dilation, and with target area SFDoing difference operation can be obtained one while containing
There is the narrowband region of target and backgroundRespective pixel set is denoted as TrimapUnknown；
S33, the region for containing only background information are represented by SB=S- (SU+SF), respective pixel set is denoted as
TrimapBackground can be obtained by a ternary mask images in this way, including well-established narrowband region SU。
3. the interactive image segmentation method of joint pixel pait according to claim 1 and super-pixel, which is characterized in that described
Step S4 the following steps are included:
S41, by target area S obtained in step S3FWith background area SBRegard virtual source node and virtual sink as respectively
Then node establishes prospect mixed Gauss model and background mixed Gauss model, Gauss using Principal Component Analysis to it respectively
Number is set as K；
S42, graph model is established to narrowband region obtained in step S3 using pixel as node, and to formula (5) energy function
In area information item and marginal information item redefined as follows:
Wherein, dist (i, j) indicates the space length of pixel i and j, IiIndicate the colouring information of pixel i, IjIndicate pixel
The colouring information of point j,8 neighborhoods are used during establishing graph model, because of λ >=b
(ιi,ιj), so the desirable λ+1 of κ=8；Parameter σ is used to control colouring information difference degree, Dx(i) indicate ith pixel point preceding
Distribution situation in scape or background model, wherein x ∈ { F, B }, value can be calculated from following formula:
Wherein,Indicate the weight shared by k-th of Gaussian Profile in the mixed Gauss model of prospect or background,WithRespectively
The mean value and covariance matrix of k-th of Gaussian Profile in expression prospect or background model,Indicate covariance matrix's
Determinant；
On the basis of good graph model established above, execute max-flow min-cut algorithm can be obtained based on Pixel-level it is global most
Excellent solution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710217696.7A CN106981068B (en) | 2017-04-05 | 2017-04-05 | A kind of interactive image segmentation method of joint pixel pait and super-pixel |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710217696.7A CN106981068B (en) | 2017-04-05 | 2017-04-05 | A kind of interactive image segmentation method of joint pixel pait and super-pixel |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106981068A CN106981068A (en) | 2017-07-25 |
| CN106981068B true CN106981068B (en) | 2019-11-12 |
Family
ID=59345558
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710217696.7A Expired - Fee Related CN106981068B (en) | 2017-04-05 | 2017-04-05 | A kind of interactive image segmentation method of joint pixel pait and super-pixel |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106981068B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108549833A (en) * | 2018-03-07 | 2018-09-18 | 江苏东大金智信息系统有限公司 | A kind of target extraction method of accurate robust |
| CN108564528A (en) * | 2018-04-17 | 2018-09-21 | 福州大学 | A kind of portrait photo automatic background weakening method based on conspicuousness detection |
| CN109559316A (en) * | 2018-10-09 | 2019-04-02 | 浙江工业大学 | A kind of improved graph theory dividing method based on super-pixel |
| CN112912921B (en) * | 2018-10-11 | 2024-04-30 | 上海科技大学 | System and method for extracting plane from depth map |
| CN110084821B (en) * | 2019-04-17 | 2021-01-12 | 杭州晓图科技有限公司 | Multi-instance interactive image segmentation method |
| CN110163869B (en) * | 2019-04-18 | 2023-01-03 | 深圳大学 | Image repetitive element segmentation method, intelligent device and storage medium |
| CN110136078A (en) * | 2019-04-29 | 2019-08-16 | 天津大学 | The semi-automatic reparation complementing method of single plant corn image leaf destruction |
| CN111161301B (en) * | 2019-12-31 | 2021-07-27 | 上海商汤智能科技有限公司 | Image segmentation method and device, electronic equipment and storage medium |
| CN111583279A (en) * | 2020-05-12 | 2020-08-25 | 重庆理工大学 | Super-pixel image segmentation method based on PCBA |
| CN111739645B (en) * | 2020-05-14 | 2024-01-30 | 北京深睿博联科技有限责任公司 | Training method of immune-related pneumonia prediction model |
| CN111932549B (en) * | 2020-06-28 | 2023-03-24 | 山东师范大学 | SP-FCN-based MRI brain tumor image segmentation system and method |
| CN114049360B (en) * | 2022-01-13 | 2022-03-22 | 南通海恒纺织设备有限公司 | Textile dyeing toner mixing control method and system based on graph cut algorithm |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102982544B (en) * | 2012-11-21 | 2015-09-30 | 清华大学 | Many foreground object image interactive segmentation method |
| US9355328B2 (en) * | 2013-03-13 | 2016-05-31 | Beihang University | Image foreground matting method based on neighborhood and non-neighborhood smoothness priors |
-
2017
- 2017-04-05 CN CN201710217696.7A patent/CN106981068B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102982544B (en) * | 2012-11-21 | 2015-09-30 | 清华大学 | Many foreground object image interactive segmentation method |
| US9355328B2 (en) * | 2013-03-13 | 2016-05-31 | Beihang University | Image foreground matting method based on neighborhood and non-neighborhood smoothness priors |
Non-Patent Citations (1)
| Title |
|---|
| 图像阈值分割关键技术研究;龙建武;《中国博士学位论文全文数据库》;20140915;第1-81页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106981068A (en) | 2017-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106981068B (en) | A kind of interactive image segmentation method of joint pixel pait and super-pixel | |
| CN110503649B (en) | Liver segmentation method based on spatial multi-scale U-net and superpixel correction | |
| CN106875406B (en) | Image-guided video semantic object segmentation method and device | |
| CN106997597B (en) | It is a kind of based on have supervision conspicuousness detection method for tracking target | |
| CN104537676B (en) | Gradual image segmentation method based on online learning | |
| Lempitsky et al. | Pylon model for semantic segmentation | |
| CN108682017B (en) | Node2Vec algorithm-based super-pixel image edge detection method | |
| CN108537239B (en) | Method for detecting image saliency target | |
| CN110838131B (en) | Method and device for realizing automatic cutout, electronic equipment and medium | |
| CN107730528A (en) | A kind of interactive image segmentation and fusion method based on grabcut algorithms | |
| CN105787948B (en) | A kind of Fast image segmentation method based on shape changeable resolution ratio | |
| CN102651128B (en) | Image set partitioning method based on sampling | |
| CN114758288A (en) | Power distribution network engineering safety control detection method and device | |
| CN103164855A (en) | Bayesian Decision Theory foreground extraction method combined with reflected illumination | |
| CN103198479A (en) | SAR image segmentation method based on semantic information classification | |
| CN109685045A (en) | A kind of Moving Targets Based on Video Streams tracking and system | |
| CN110956158A (en) | Pedestrian shielding re-identification method based on teacher and student learning frame | |
| CN102842125A (en) | Image processing apparatus and image processing method | |
| CN111798470A (en) | Crop image entity segmentation method and system applied to intelligent agriculture | |
| CN104021560A (en) | Image segmentation method based on multi-objective evolutionary algorithm | |
| CN109191485B (en) | Multi-video target collaborative segmentation method based on multilayer hypergraph model | |
| CN107610137A (en) | A kind of high-resolution remote sensing image optimal cut part method | |
| Grady et al. | An energy minimization approach to the data driven editing of presegmented images/volumes | |
| Li et al. | Saliency detection via alternative optimization adaptive influence matrix model | |
| CN102184524B (en) | Neighbourhood learning culture gene image segmentation method based on standard cut |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20191112 |