CN101520894B - Method for extracting significant object based on region significance - Google Patents

Abstract

The invention discloses a method for extracting a significant object based on region significance. The method comprises the following steps: firstly, establishing a significant image with constant scaThe invention discloses a method for extracting a significant object based on region significance. The method comprises the following steps: firstly, establishing a significant image with constant scale through by calculating the multiresolution contrast feature of an input image, and dividing the input image into different regions by a non-parameter nuclear density evaluation method; secondly, anle through by calculating the multiresolution contrast feature of an input image, and dividing the input image into different regions by a non-parameter nuclear density evaluation method; secondly, and then, calculating specific values of region significance of each region assembly and a complementary set thereof; and finally, extracting the significant object through by acquiring the maximum valud then, calculating specific values of region significance of each region assembly and a complementary set thereof; and finally, extracting the significant object through by acquiring the maximum value in the specific values, which comprises the following steps: (1) inputting the an image, and establishing the a significant image with constant scale; (2) inputting the image to realize image segmene in the specific values, which comprises the following steps: (1) inputting the an image, and establishing the a significant image with constant scale; (2) inputting the image to realize image segmentation; and (3) extracting the significant image. The method is combined with the region significance, not only can accurately extract a single significant object, but also can extract a plurality oftation; and (3) extracting the significant image. The method is combined with the region significance, not only can accurately extract a single significant object, but also can extract a plurality ofsignificant objects, so that the extracted significant object can meet the requirement of human vision, and can improve the accuracy of segmentation.significant objects, so that the extracted significant object can meet the requirement of human vision, and can improve the accuracy of segmentation.

Description

Remarkable object extraction method based on region significance

Technical field

The present invention relates to a kind of method of Computer Image Processing, specifically relate to a kind of image partitioning method.

Background technology

Image segmentation is a major issue of graphical analysis, pattern-recognition and computer vision field, also is a difficult point problem simultaneously, and the final purpose of image segmentation is to be partitioned into the object with particular real-world meaning, i.e. semantic object.The method that has is used the remarkable object in discernible high-level information (for example people's face and text) and the image conspicuousness positioning image.But because there is not discernible high-level information in some images, even perhaps exist discernible high-level information to be difficult to extract automatically, so its use is limited yet.The image conspicuousness is always available, but it can not provide enough information for the remarkable object in the positioning image.Rudimentary spatial feature not necessarily well mates remarkable object.For example, the edge of some high-contrast between the image-region is normally noticeable, will be identified as remarkable object, and in fact not be correct remarkable object.In addition, existing image conspicuousness detection method can not be identified in the specific image feature that occurs on the different graphical rules.Do not solve the constant conspicuousness problem of graphical rule, can lose the notable feature on some graphical rules, cause the image segmentation effect not fully up to expectations.

Summary of the invention

The objective of the invention is to overcome the deficiencies in the prior art a kind of remarkable object extraction method based on region significance is provided, this method can accurately extract a plurality of remarkable objects, improve the accuracy of image segmentation, make segmentation result satisfy the human eye vision requirement.

For solving the problems of the technologies described above, the technical solution adopted in the present invention is: above-mentioned remarkable object extraction method based on region significance, at first, multiresolution contrast metric by calculating input image, set up a conspicuousness image that yardstick is constant, input picture is divided into different zones with the norm of nonparametric kernel density method of estimation; Then, calculate the ratio of the region significance of each zone combination and its supplementary set; At last, extract remarkable object by the maximal value of getting this ratio, concrete steps are as follows:

1, input picture, set up the constant conspicuousness image of yardstick:

1., input picture is converted to L ^*a ^*The b color space;

2., use formula (1), set up the Gaussian image pyramid

$I_l(i,j)=\underset{m=-2}{\overset{2}{\mathrm{\Σ}}}\underset{n=-2}{\overset{2}{\mathrm{\Σ}}}w(m,n)I_{l-1}(2i+m,2j+n)---\left(1\right)$

In the formula, 0＜l＜N, 0≤i＜C _l, 0≤j＜R _l, the pyramidal progression N=log of Gaussian image ₂(min (W, H)/10), W and H are respectively the wide and high of original image, C _lAnd R _lBe respectively the picture size of l level, (m n) is weight to w;

3., use formula (2), (3), thereby set up the contrast image pyramid, the contrast value C on graphical rule l by on each graphical rule, calculating contrast image _{I, j, l}Be defined in pixel on the yardstick l (i, j) and the weighted sum of the difference between its neighborhood territory pixel, promptly

$C_{i,j,l}=\underset{q\∈\mathrm{\Θ}}{\mathrm{\Σ}}{w}_{i,j,l}D(p_{i,j,l},p_q)---\left(2\right)$

w _i，j，l＝1-r _i，j，l/r _l，max (3)

In the formula, Θ is pixel (i, neighborhood territory pixel set j), the p on yardstick l _{I, j, l}Be pixel (i, color j), the p on yardstick l _qBe p _{I, j, l}The color of pixel of neighborhood territory pixel set, D is to use the colour-difference of Euclidean distance.Weight factor w _{I, j, l}Be used to illustrate that the center of an image visually is more significant usually.r _{I, j, l}Be from (i, j) to the distance of picture centre, r _{L, max}Be ultimate range to picture centre;

4., use formula (4) that the contrast image on all yardsticks is adjusted to the size the same with original image, then their are set up the conspicuousness image mutually,

$I_{l,k}(i,j)=\underset{m=-2}{\overset{2}{\mathrm{\&Sigma;}}}\underset{n=-2}{\overset{2}{\mathrm{\&Sigma;}}}w(m,n)I_{l,k-1}(\frac{i-\mathrm{<figure-callout\; id="2"\; label="m"\; filenames="H2009100462762E00022.png"\; state="\{\{state\}\}">m</figure-callout>}}{2},\frac{j-n}{2})---\left(4\right)$

0＜l in the formula＜N, 0≤k, 0≤i＜C _L-k, 0≤j＜R _L-k, I _{L, k}(i is I j) _lEnlarge k result doubly.

2, input picture is realized image segmentation

At first, image with input, use formula (5), (6), utilize the image self-information, obtain representative seed points, and use the seed points quantized image, then, removing and use formula (7), (8) that the zone is carried out in the low zone of conspicuousness according to the area in zone to noise more among the quantification figure merges, realizes the Region Segmentation of image, be the formula of nonparametric density Estimation below

$f\left(x\right)=\frac{1}{U}\underset{i=1}{\overset{U}{\mathrm{\&Sigma;}}}{K}_{\mathrm{\&sigma;}}(x-x_i)---\left(5\right)$

$K_{\mathrm{\&sigma;}}\left(x\right)=\frac{1}{\sqrt{2\mathrm{\&pi;}{\mathrm{\&sigma;}}^2}}{e}^{-{\left|\right|x\left|\right|}^2/2{\mathrm{\&sigma;}}^2}---\left(6\right)$

Density f (x) value is the convolution value of feature point set and kernel function.U is all the pixel numbers in the image.K (x) is a kernel function, and wherein x is the unique point that will calculate.σ is a frequency range.

The significant indexes in zone is used for that each zone causes the probability size that human eye is paid close attention in the presentation video.Utilize each regional area shared ratio and this zone in having identical seed points color region to obtain regional significant indexes in the shared ratio of entire image.If certain zone shared ratio in all same color zones is less, significant indexes that so should the zone is just lower.The significant indexes formula definition in zone is

$S\left({R^i}_j\right)=\left(\frac{N_{{R^i}_j}}{\underset{j=1}{\overset{m_j}{\mathrm{\&Sigma;}}}{N}_{{R^i}_j}}\right)\&times;\left(\frac{N_{{R^i}_j}}{\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}\underset{j=1}{\overset{m_j}{\mathrm{\&Sigma;}}}{N}_{{R^i}_j}}\right)---\left(7\right)$

R in the formula _j ⁱRepresent j zone in i the seed points color, S (R _j ⁱ) be R _j ⁱSignificant indexes,

Be R _j ⁱNumber of pixels,

Be the number of pixels summation of All Ranges in i the seed points color,

Number summation for all pixels in the image.

After having obtained regional significant indexes, need to remove some non-significant zones.As shown in Figure 5, if non-marking area is only adjacent with a zone, then can be merged by adjacent area.If removed zone is positioned at the intersection in several zones, then need the acting force between the zoning to handle, see shown in the formula (8)

F _ij＝k _ijA(R _i)A(R _j) (8)

K in the formula _IjBe the acting force factor, A (R _i), A (R _j) be respectively each regional area, F _IjBe the gravitation between i and two zones of j, can further it be divided into two parts, i.e. F _{I ← j}And F _{J ← i}, F wherein _{I ← j}Be the adjacent area j in i zone contribution degree to its acting force, on the contrary F _{J ← i}Be the adjacent area i in j zone contribution degree to its acting force.The acting force of both direction is because of k _IjDifference and difference.

Remarkable object may be that an independent zone among the Region Segmentation figure may be the set in several zones also, supposes that E is the set of All Ranges in the image, if V zone arranged in the image, then the number of the potential object of the candidate among the E is 2 ^VBut whether each subclass can both become remarkable object, and common one entire image is not as a remarkable object significantly to liking an integral body, and significantly object can not be sky.Suppose that C ∈ E is a remarkable object, it must satisfy following three conditions:

①C≠{}；

②C≠{1，2，3...，V}；

If 3. C comprises a more than zone, these zones must interconnect so, and the object of extraction must be an integral body.

Obtain C, also can obtain its supplementary set C.Zone in the supplementary set must be adjacent with corresponding region among the C.

3, remarkable object extraction

After obtaining above-mentioned zone combination and supplementary set thereof, use formula (9), (10) and (11) to calculate their conspicuousness value and ratio.When the difference of subject area and its peripheral region is big, can think this to as if significant, so the question resolves itself into that extracts a remarkable object finds an effectively zone combination C _i, it makes T _Div(C) reach maximal value.

The formula of the conspicuousness value of zoning combination and supplementary set thereof:

T(C)＝S(C)/A(C) (9)

T(C)＝S(C)/A(C) (10)

S in the formula (C) and S (C) be respectively C and C all pixels the conspicuousness value and, A (C) and A (C) are respectively the areas of C and C; When the difference of subject area and its peripheral region is big, then think this to as if significant,

Calculate the ratio of T (C) and T (C):

T _div(C)＝T(C)/T(C) (11)

So the question resolves itself into that extracts a remarkable object finds an effectively zone combination C _i, it makes T _Div(C) reach maximal value, promptly

C _i＝argmax[T _div(C)] (12)

Significantly the object extraction step is as follows:

(1) calculate the conspicuousness value T (C) and the T (C) of each zone combination and supplementary set thereof, and their ratio T _Div(C).Find one to make T _Div(C) reach the combination of peaked zone;

(2), extract C if do not satisfy end condition _iAs remarkable object, delete C then _i, execution in step (1); Otherwise execution in step (3);

(3) if satisfy any one end condition, terminator,

End condition has:

1. A (C)＜λ A, A is the area of original image in the formula,

②cpt(C)＜μ，

cpt(C)＝A(C)/(H(C)*W(C)) (13)

Cpt in the formula (C) is the tight ness rating of zone combination C, and H (C) and W (C) are respectively the height and width of C.

3. only remain a zone,

Otherwise, execution in step (2).

Remarkable object extraction method based on region significance of the present invention has following advantage compared with prior art: this method calmodulin binding domain CaM conspicuousness, can not only extract single remarkable object accurately, and can accurately extract a plurality of remarkable objects, make the remarkable object of extraction satisfy the human eye vision requirement, and can improve the accuracy of cutting apart.

Description of drawings

Fig. 1 is the process flow diagram of the remarkable object extraction based on region significance of the present invention;

Fig. 2 is the process flow diagram of image segmentation;

Fig. 3 quantizes rear region figure;

Fig. 4 schemes after eliminating the zonule;

Fig. 5 is the areal map after merging;

Fig. 6 is that the zone merges process flow diagram;

Fig. 7 is the extraction diagram to image, and provides the result schematic diagram of step;

Fig. 8 is the extraction result diagram of other types of image.

Embodiment

Embodiment to the remarkable object extraction method that the present invention is based on region significance is described in further detail below in conjunction with accompanying drawing.

Fig. 1 is a process flow diagram of the present invention, at first, obtains the conspicuousness image that yardstick is constant by Gaussian image pyramid and the contrast image pyramid of setting up input picture, with the norm of nonparametric kernel density method of estimation input picture is divided into different zones; Calculate the ratio of the region significance of each zone combination and its supplementary set then; Extract remarkable object by the maximal value of getting this ratio at last.The specific implementation process is as follows:

1, input picture, set up the constant conspicuousness image of yardstick:

1., input picture is converted to L ^*a ^*The b color space;

2., use formula (1) to set up the Gaussian image pyramid;

3., use formula (2), (3) thus on each graphical rule, calculate contrast image and set up the contrast image pyramid;

4., use formula (4) that the contrast image on all yardsticks is adjusted to the size the same with original image, then their are set up the conspicuousness image mutually.

2, input picture is realized image segmentation

As shown in Figure 2, at first, by calculating the original image of input, use formula (5), (6), obtain representative seed points, and use the seed points quantized image, then, removing and use formula (7), (8) that the zone is carried out in the low zone of conspicuousness according to the area in zone to noise more among the quantification figure merges, realizes the Region Segmentation of image.

(1) image quantization

As shown in Figure 2, at first, convert L to by the original image that calculates input ^*a ^*The b color space, and add up the pixel quantity of three Color Channels; Then, use formula (5), (6), statistic being examined the probability density of estimation calculates, obtain the gray level probability density value of three passages, be convenient to use subsequently the gradient rise method to obtain the local extremum of three passages, utilize permutation and combination to obtain seed points again, each pixel value in the image and seed points are done color distance calculating, select immediate seed points to replace original pixel value, finish the quantification of image.

Occurred a lot of scattered noises among the quantification figure, as shown in Figure 3, according to region significance, the conspicuousness that remarkable object should have and the requirement of integrality are not satisfied in these scattered zonules, so it must be eliminated.Here mainly utilized regional area to judge, if the area in zone then is the zone that will be eliminated less than certain critical value.Rule of thumb setting critical value is 0.5% of the total area.Region area then is eliminated less than this value, and be about to this zone and merge to contiguous zone, as shown in Figure 4, the noise region that the above-mentioned step of process just can be eliminated small size on the above-mentioned image.But divided area still can not be fully and the condition of marking area coincide, therefore next need to carry out zone merging so that marking area is more complete.

(2) zone merges

The zone merges flow process and sees Fig. 6.After the less noise region removal in the above-mentioned quantized image, tend to occur the zone that there is identical seed points color value many places in the image.Visually human eye is difficult to identical attention rate is all expressed in the zone of the many places color similarity that occurs in the piece image, so need to handle this problem.As shown in Figure 4, zone 1,2,3,4,5 is arranged in the image, zone 3,5 two zones with same color, zone have wherein appearred, and in entire image, so because the attractive force of the 3 pairs of human eyes in zone is obviously lower it is eliminated, utilizes regional significant indexes whether each zone is eliminated below and judge.

The significant indexes in zone is used for that each zone causes the probability size that human eye is paid close attention in the presentation video.Utilize each regional area shared ratio and this zone in having identical seed points color region to obtain regional significant indexes in the shared ratio of entire image.If certain zone shared ratio in all same color zones is less, significant indexes that so should the zone is just lower.The significant indexes formula in zone is shown in (7).

Use an adjustable critical value to remove the zone with less significant indexes, critical value setting is that the maximal value of significant indexes is multiplied by certain ratio, and the ratio of choosing here is 0.6%.

After having obtained regional significant indexes, need to remove some non-significant zones.As shown in Figure 5, if non-marking area is only adjacent with a zone, then can be merged by adjacent area.If removed zone is positioned at the intersection in several zones, then need the acting force between the zoning to handle, see shown in the formula (8).

During gravitation between the zoning, the adjacent areas area big gravitation of healing is also just big, otherwise then little.If region area is big but the different degree height of color then can't produce enough gravitation.For being merged the zone of depending on the gravitation maximum by which zone.Repeated calculation up to the significant indexes of All Ranges greater than critical value, thereby obtain the Region Segmentation figure of final image.

Remarkable object may be that an independent zone among the Region Segmentation figure may be the set in several zones also, supposes that E is the set of All Ranges in the image, if V zone arranged in the image, then the number of the potential object of the candidate among the E is 2 ^VAs V=3,8 subclass are arranged among the E then:

{}，{1}，{1，2}，{1，2，3}，{1，3}，{2}，{2，3}，{3}

But whether each subclass can both become remarkable object, and common one entire image is not as a remarkable object significantly to liking an integral body, and significantly object can not be sky.Suppose that C ∈ E is a remarkable object, it must satisfy following three conditions:

①C≠{}；

②C≠{1，2，3...，V}；

If 3. C comprises a more than zone, these zones must interconnect so, and the object of extraction must be an integral body.

Obtain C, also can obtain its supplementary set C.Zone in the supplementary set must be adjacent with corresponding region among the C.

3, remarkable object extraction

After having obtained regional combination and supplementary set thereof, use formula (9), (10) and (11) to calculate their region significance value and ratio.When the difference of subject area and its peripheral region is big, can think this to as if significant, so the question resolves itself into that extracts a remarkable object finds an effectively zone combination C _i, it makes T _Div(C) reach maximal value.

Significantly the object extraction step is as follows:

(1) calculate the conspicuousness value T (C) and the T (C) of each zone combination and supplementary set thereof, and their ratio T _Div(C).Find one to make T _Div(C) reach peaked zone combination C _i

(2), extract C if do not satisfy end condition _iAs remarkable object, delete C then _i, execution in step (1); Otherwise execution in step (3);

(3) if satisfy any one end condition, terminator,

End condition has:

1. A (C)＜λ A, 2. 3. cpt (C)＜μ only remains a zone,

Otherwise, execution in step (2).

Set λ and μ and be respectively 0.01 and 0.2.

The present invention carries out emulation experiment, and as shown in Figure 7 and Figure 8, they are to be that programming realizes on the PC test platform of 2.0GHz, internal memory 512M at CPU, wherein, the experimental result picture that Fig. 7 provides has comprised the several important steps in the algorithm, (a) is original image, and a safflower wherein is remarkable object; (b)-(f) be the experimental result of step 1, (b)-(e) be respectively the conspicuousness image on four different images yardsticks from thin yardstick to thick yardstick, on thinner graphical rule, the profile that safflower and part greenery are arranged (b), and on thicker graphical rule, (e) having only safflower in substantially, (f) is the constant conspicuousness image of yardstick; (g) being Region Segmentation figure after merging, is the experimental result of step 2, and safflower and greenery etc. are divided into different zones; (h) being final remarkable object extraction figure, is the experimental result of step 3, and safflower is extracted accurately as remarkable object.

Fig. 8 then is the experimental result that provides all the other types, in order to practicality of the present invention and algorithm accuracy to be described.Wherein as shown in Figure 8, this figure has animal 3 width of cloth, car 1 width of cloth, view 1 width of cloth and spend 1 width of cloth, be divided into 6 row four row and arrange, first row are original images, secondary series, the 3rd row and the 4th row be respectively the original image from first row extract first, the image of second and the 3rd remarkable object.

From the result of the foregoing description as can be seen, the present invention can not only extract single remarkable object accurately, and can accurately extract a plurality of remarkable objects, satisfies the vision requirement of human eye simultaneously.

Claims (1)

1. remarkable object extraction method based on region significance, it is characterized in that at first, by the multiresolution contrast metric of calculating input image, set up a conspicuousness image that yardstick is constant, with the norm of nonparametric kernel density method of estimation input picture is divided into different zones; Then, calculate the ratio of the region significance of each zone combination and its supplementary set; At last, extract remarkable object by the maximal value of getting this ratio, its concrete steps are as follows:

(1), input picture, set up the constant conspicuousness image of yardstick;

(2), input picture, realize image segmentation;

(3), remarkable object extraction;

Above-mentioned steps (1) input picture, the step of setting up the constant conspicuousness image of yardstick comprises:

1., input picture is converted to the L*a*b color space;

2., use formula (1), set up the Gaussian image pyramid

In the formula, 0＜l＜N, 0≤i＜C _l, 0≤j＜R _l, the pyramidal progression N=log of Gaussian image ₂(min (W, H)/10), W and H are respectively the wide and high of original image, C _lAnd R _lBe respectively the picture size of l level, (m n) is weight to w;

3., use formula (2), (3), thereby set up the contrast image pyramid, the contrast value C on graphical rule l by on each graphical rule, calculating contrast image _{I, j, l}Be defined in pixel on the yardstick l (i, j) and the weighted sum of the difference between its neighborhood territory pixel, promptly

w _i，j，l＝1-r _i，j，l/r _l，max (3)

In the formula, Θ is pixel (i, neighborhood territory pixel set j), the p on yardstick l _{I, j, l}Be pixel (i, color j), the p on yardstick l _qBe p _{I, j, l}The color of pixel of neighborhood territory pixel set, D is to use the colour-difference of Euclidean distance, weight factor w _{I, j, l}Be used to illustrate that the center of an image visually is more significant usually, r _{I, j, l}Be from (i, j) to the distance of picture centre, r _{L, max}Be ultimate range to picture centre;

4., use formula (4) that the contrast image on all yardsticks is adjusted to the size the same with original image, then their are set up the conspicuousness image mutually,

0＜l in the formula＜N, 0≤k, 0≤i＜C _L-k, 0≤j＜R _L-k, I _{L, k}(i is I j) _lEnlarge k result doubly.

Above-mentioned steps (2) input picture, the step that realizes image segmentation comprises: at first, with the image of input, use formula (5), (6), utilize the image self-information, obtain representative seed points, and use the seed points quantized image, then, remove and use formula (7), (8) that the zone is carried out in the low zone of conspicuousness according to the area in zone to noise more among the quantification figure and merge, realizing the Region Segmentation of image, is the formula of nonparametric density Estimation below

Density f (x) value is the convolution value of feature point set and kernel function, and U is all the pixel numbers in the image, and K (x) is a kernel function, and wherein x is the unique point that will calculate, and σ is a frequency range, and the significant indexes formula in zone is:

In the formula

Represent j zone in i the seed points color,

For

Significant indexes, For Number of pixels,

Be the number of pixels summation of All Ranges in i the seed points color,

Be the number summation of all pixels in the image, see shown in the formula (8)

F _ij＝k _ijA(R _i)A(R _j) (8)

K in the formula _IjBe the acting force factor, A (R _i), A (R _j) be respectively each regional area, F _IjGravitation between i and two zones of j further is divided into it two parts, i.e. F _{I ← j}And F _{J ← i}, F wherein _{I ← j}Be the adjacent area j in i zone contribution degree to its acting force, on the contrary F _{J ← i}Be the adjacent area i in the j zone contribution degree to its acting force, the acting force of both direction is because of k _IjDifference and difference, repeated calculation up to the significant indexes of All Ranges greater than critical value, thereby obtain the Region Segmentation figure of final image.

Described step (3) is the object extraction step significantly: after obtaining above-mentioned zone combination and supplementary set thereof, use formula (9), (10) and (11) to calculate their conspicuousness value and ratio, the question resolves itself into that extracts a remarkable object finds an effectively zone combination C _i, it makes T _Div(C) reach maximal value, the formula of the conspicuousness value of zoning combination and supplementary set thereof:

T(C)＝S(C)/A(C) (9)

S in the formula (C) and

Be respectively C and

All pixels the conspicuousness value and, A (C) and

Be respectively C and Area; Calculate T (C) and

Ratio:

The question resolves itself into that extracts a remarkable object finds an effectively zone combination C _i, it makes T _Div(C) reach maximal value, promptly

C _i＝argmax[T _div(C)] (12)

Significantly the object extraction step is as follows:

(1), calculate each zone combination and supplementary set thereof conspicuousness value T (C) and

And their ratio T _Div(C), find one to make T _Div(C) reach peaked zone combination C _i

(2), extract C if do not satisfy end condition _iAs remarkable object, delete C then _i, execution in step (1); Otherwise execution in step (3);

(3) if satisfy any one end condition, stop extracting, end condition has:

1. A (C)＜λ A, A is the area of original image in the formula;

②cpt(C)＜μ，

cpt(C)＝A(C)/(H(C)*W(C)) (13)

Cpt in the formula (C) is the tight ness rating of zone combination C, and H (C) and W (C) are respectively the height and width of C;

3. only remain a zone, otherwise, execution in step (2).

Images