CN106780450A - A kind of image significance detection method based on low-rank Multiscale Fusion - Google Patents

Abstract

The present invention relates to a kind of image significance detection method based on low-rank Multiscale Fusion, its technical characterstic includes：Image to being input into carries out single scale conspicuousness detection；Image after being detected to single scale conspicuousness carries out multiple dimensioned conspicuousness fusion treatment, obtains merging Saliency maps；Conspicuousness micronization processes are carried out to the fusion Saliency maps after multiple dimensioned conspicuousness fusion treatment, final collaboration Saliency maps picture is obtained.Be applied to the method for the conspicuousness detection method recovered based on low-rank matrix and the fusion of multiple dimensioned conspicuousness in conspicuousness detection by the present invention, and by with the collaboration conspicuousness priori based on GMM, the detection of multiple dimensioned low-rank conspicuousness is generalized in multiple image collaboration conspicuousness detection, to detect the same or analogous region occurred in multiple image, solve the problems, such as that scale selection is difficult, more reliable conspicuousness testing result is achieved, helps further to improve the disposal ability of conspicuousness detection.

Description

A kind of image significance detection method based on low-rank Multiscale Fusion

Technical field

It is aobvious the invention belongs to Computer Vision Detection Technique field, especially a kind of image based on low-rank Multiscale Fusion Work property detection method.

Background technology

In computer vision field, conspicuousness object detecting method is divided into bottom-up scene drive model and Zi Ding The downward major class of expectation driving model two.Bottom-up method is based primarily upon the scene information of picture scenery, and top-down Method be then by knowledge, expect and purpose determine.Many conspicuousness detection methods, such as RC, CA have been proposed now Deng.The most of conspicuousness both for single scale picture of these conspicuousness detection methods is detected and had been achieved for good Effect.But it is exactly when object is in the natural scene of the big contrast of small yardstick that these methods have a common problem When, typically can not well detect the conspicuousness object in picture.For such case, typically there are two kinds of solutions, one Plant and be to continue with finding more preferable conspicuousness object, another kind is auxiliary using other also pictures comprising identical conspicuousness object Monitoring conspicuousness object is helped, this method is referred to as collaboration conspicuousness detection.

The conspicuousness detection method recovered based on low-rank matrix is based on following a priori assumption：Conspicuousness target is in view picture figure On be it is sparse, such piece image can just regard as background plus in background sparse distribution some conspicuousness targets, And image background has low-rank characteristic, and then a secondary natural image is decomposed into a low-rank matrix and a sparse matrix, because The detection of this conspicuousness is converted into a recovery problem for low-rank matrix.

The content of the invention

It is an object of the invention to overcome the deficiencies in the prior art, there is provided a kind of image based on low-rank Multiscale Fusion shows Work property detection method, solves the problems, such as that existing detection method has scale selection difficulty and reliability.

The present invention solves its technical problem and takes following technical scheme to realize：

A kind of image significance detection method based on low-rank Multiscale Fusion, comprises the following steps：

Step 1, the image to being input into carry out single scale conspicuousness detection；

Step 2, single scale conspicuousness is detected after image carry out multiple dimensioned conspicuousness fusion treatment, obtain fusion notable Property figure；

Step 3, conspicuousness micronization processes are carried out to the fusion Saliency maps after multiple dimensioned conspicuousness fusion treatment, obtained most Whole collaboration Saliency maps picture.

Further, the specific processing method of the step 1 is comprised the following steps：

(1) image is too cut into multi-scale division figure and feature extraction is carried out；

(2) conspicuousness priori treatment is carried out using background transcendental method；

(3) conspicuousness calculating is carried out.

Further, step method (1) is：By the image for being input into, the image point that will be input into using SLIC methods Super-pixel is cut into, and extracts position feature, color characteristic and the textural characteristics of 122 dimensions.

Further, step conspicuousness computational methods (3) are carried out using following conspicuousness model：

SP (i) is i-th significance value of super-pixel,It is the significance value of i-th super-pixel, j-th feature.It is i-th significance value vector of all features of super-pixel.

Further, the specific method of the step 2 is：First, piece image is divided into different yardsticks；Then, count Calculate the Saliency maps on each yardstick；Finally, counted by the way that the significance value of all yardsticks is multiplied by into corresponding adaptive weighting Calculate fusion Saliency maps.

Further, the adaptive weighting is expressed as follows：

Wherein, Z is a partition function；

The fusion Saliency maps are calculated using equation below：

ω i represent i-th adaptive weighting of the Saliency maps of yardstick,Represent i-th feature of yardstick Value,Represent the Saliency maps after multiple yardstick fusions.

Further, the processing method of the step 3 includes：

(1) smooth disposal is carried out to present image so that image reaches space smoothing；

(2) collaboration conspicuousness detection is carried out to image.

Further, (1) the step is to the method that present image carries out smooth disposal：Using following energy function reality It is existing：

Wherein, S_IThe significance value of each super-pixel i is represented,The probability of background is represented,Expression prospect it is general Rate,Nei(i)：Represent i-th neighborhood of super-pixel, weights omega_ij：It is defined as：

Wherein,The L2 distances of the color average in CIE-LAB color spaces are represented,

Further, (2) the step carries out cooperateing with conspicuousness detection to comprise the following steps to image：

1. single significant point detection：For a series of image I for giving_set={ I₁, I₂..., I_n, calculate every width figure The single Saliency maps of picture, use S_iRepresent i-th single Saliency maps of image；

2. binary segmentation：Use adaptive threshold T_i：Single Saliency maps are divided into binary mask M_i, T_iIt is defined as：

T_i=α mean (S_i)

Wherein, α=2；

3. conspicuousness prior estimate is cooperateed with：GMM algorithms are the foreground pixel in i-th picture using 5 Gauss models Build color model G_i, then with the mould M estimated in j-th picture_jProspect probability；It is general n prospect to be obtained for each picture Then every pictures are calculated collaboration conspicuousness priori to obtain the average value of these estimates by the estimate of rate；

4. collaboration conspicuousness is calculated：Collaboration conspicuousness priori is merged into single conspicuousness detection model and obtains last Collaboration Saliency maps picture.

Advantages and positive effects of the present invention are：

The method application that the present invention merges the conspicuousness detection method recovered based on low-rank matrix and multiple dimensioned conspicuousness Arrive in conspicuousness detection, and by with the collaboration conspicuousness priori based on GMM, multiple dimensioned low-rank conspicuousness being detected and being promoted To in multiple image collaboration conspicuousness detection, to detect the same or analogous region occurred in multiple image, the present invention is carried The multiple dimensioned super-pixel blending algorithm of low-rank for going out solves the problems, such as that scale selection is difficult, achieves more reliable conspicuousness detection As a result, contribute to further to improve the disposal ability that conspicuousness is detected.

Brief description of the drawings

Fig. 1 is the image significance detection method flow chart based on low-rank Multiscale Fusion of the invention；

Fig. 2 is the significant characteristics dimension that the present invention is extracted and description schematic diagram；

Fig. 3 is the comparative effectiveness figure of present invention performance on MSRA data sets；

Fig. 4 is the comparative effectiveness figure of present invention performance on ESCCD data sets；

Fig. 5 is the comparing figure of collaboration conspicuousness method performance of the present invention on image pair data sets.

Specific embodiment

The embodiment of the present invention is further described below in conjunction with accompanying drawing：

A kind of image significance detection method based on low-rank Multiscale Fusion, as shown in figure 1, comprising the following steps：

Step 1, the image to being input into carry out the conspicuousness detection of single scale.Specific method is：

(1) image is too cut into multi-scale division figure and feature extraction is carried out

For the image being input into, we are divided into super-pixel using SLIC, and extract the feature of 122 dimensions, including Position, color, texture, as shown in Figure 2.Specific practice is：We extract the color characteristic of 40 dimensions, lower 4 directions of 3 yardsticks totally 12 Individual steerable pyramids features, the direction of 3 yardstick 12 totally 36 Gabor characteristics, the feature of 31 dimensions is extracted with HOG.

(2) conspicuousness priori treatment

At present, some top-down methods are already used to further improve the performance of conspicuousness detection.It is representational Method has a variety of conspicuousness priori, and such as center priori, object priori, background priori, these methods are provided to improve Conspicuousness object position that may be present in piece image.In the method, we carry out conspicuousness using background transcendental method Priori treatment.

(3) conspicuousness is calculated

Due to low rank analysis to conspicuousness detect it is helpful, we can piece image be divided into redundancy section and significantly Property part.Redundancy section represents the systematicness with height, and conspicuousness part represents novelty.We can be this breakdown It is shown as the recovery problem of low-rank matrix：

S.t.F=B+S

Wherein, F=[f₁,f₂,...,f_n] be N number of characteristic vector composition eigenmatrix, B is obtained by background modeling The low-rank matrix for arriving, S is to model the sparse matrix for obtaining by conspicuousness.

Problem due to more than is a np problem, therefore we are converted into following mode to solve：

S.t.F=B+S

But, poor object conspicuousness testing result can be always obtained in initial Feature Space Decomposing F.In order to obtain One good result, we first learn a transformation matrix T, by the way that by eigenmatrix F premultiplication T, it is special that we obtain a conversion Levy matrix TF.In space after the conversion, the feature of image background is present in a low dimensional subspace.Therefore, they can To be expressed as a low-rank matrix.Priori P can be updated with P premultiplications TF, therefore, final conspicuousness model is：

S.t.TFP=B+S

Assuming that S is the optimal solution of equation.Significance value SP (i) of so i-th super-pixel is：

Step 2, single scale conspicuousness is detected after image carry out multiple dimensioned conspicuousness fusion treatment, obtain fusion notable Property figure.

It is reliable aobvious in order to obtain one because the conspicuousness Detection results on single scalogram picture may be undesirable Work property testing result, this patent Multiscale Fusion method：First, piece image is divided into different yardsticks by we；Then, use Above method calculates the Saliency maps on each yardstick；Finally, we are corresponding by the way that the significance value of all yardsticks is multiplied by Adaptive weighting calculates fusion Saliency maps.

One significance value of super-pixel is just included in the average value of all significance value in this super-pixel region, The significance value of all super-pixel on each yardstick is expressed as a row vector by we, each super-pixel on all yardsticks Significance value constitute a conspicuousness oriental matrix SI.Ideally conspicuousness testing result is all on all yardsticks Consistent, therefore, the order of oriental matrix should be 1.We can be converted into this problem the recovery problem of low-rank matrix：

S.t.SI=L+E

Wherein, optimal solution E represents the difference of multiple dimensioned conspicuousness testing result.We are absolute the every row element in E Value summation, will obtain a vectorWherein, n represents yardstick.E_iIt is bigger, represent i-th it is notable Property figure is higher with the inconsistency of other Saliency maps.Therefore, corresponding Saliency maps should assign a weight for very little.Adapt to Property weight is expressed as：

Wherein, Z is a partition function.Finally, fusion Saliency maps can be calculated with equation below：

Step 3, conspicuousness micronization processes are carried out to the fusion Saliency maps after multiple dimensioned conspicuousness fusion treatment, obtained most Whole Saliency maps picture.Specifically include：

(1) smooth disposal is carried out to present image so that image reaches space smoothing

After completing, we start to consider the flatness between neighbouring super pixels.We are with an energy function come excellent Change the Saliency maps after fusion：

Wherein, S_IThe significance value of each super-pixel i is represented,The probability of background is represented,The probability of expression prospect. (in the method), Nei (i)：Represent i-th neighborhood of super-pixel.Weights omega_ij：It is defined as：

Wherein,The L2 distances of the color average in CIE-LAB color spaces are represented,

(2) collaboration conspicuousness detection, including the detection of single significant point, binary segmentation, collaboration conspicuousness priori are carried out to image Estimate, collaboration conspicuousness calculates process step, be described as follows：

1. single significant point detection

For a series of image I for giving_set={ I₁, I₂..., I_n, calculate every width figure with method mentioned above The single Saliency maps of picture, use S_iRepresent i-th single Saliency maps of image.

2. binary segmentation

We use adaptive threshold T_i：Single Saliency maps are divided into binary mask M_i, T_iIt is defined as：

T_i=α mean (S_i)

Wherein, α=2 in our experiment.The significance value pixel bigger than the adaptive threshold that we give or super picture Element is exactly prospect, is otherwise exactly background.

3. conspicuousness prior estimate is cooperateed with

We obtain collaboration conspicuousness priori using GMM, and specific method is：GMM algorithms using 5 Gauss models come for Foreground pixel in i-th picture builds color model G_i, then with the mould M estimated in j-th picture_jProspect probability.For Each picture will obtain the n estimate of prospect probability, then calculate collaboration conspicuousness priori to every pictures to obtain this The average value of a little estimates.

4. collaboration conspicuousness is calculated

Finally, we are merged into collaboration conspicuousness priori in single conspicuousness detection model that to obtain last collaboration notable The image of property.

The multiple dimensioned super-pixel of low rank analysis can be realized by above step to detect obvious object function.

Fig. 3 is the comparative effectiveness figure of present invention performance on MSRA data sets, it can be seen that on the data set Compared with prior art, the PR curves and ROC curve of our methods be it is optimal, MAE for minimum, AUC highests；Fig. 4 is this hair The comparative effectiveness figure of the bright performance on ESCCD data sets, it can be seen that, on the data set compared with prior art The PR curves and ROC curve of our methods be it is optimal, MAE for minimum, AUC highests；Fig. 5 is the present invention in image pair numbers According to the comparing figure of the collaboration conspicuousness method performance on collection, it can be seen that, on the data set compared with prior art The fmeasure of our methods, precision and recall are highest, and MAE is minimum；Therefore the present invention is in different data On collection compared with prior art, its Detection results is significantly increased.

It is emphasized that embodiment of the present invention is illustrative, rather than limited, therefore present invention bag The embodiment for being not limited to described in specific embodiment is included, it is every by those skilled in the art's technology according to the present invention scheme The other embodiment for drawing, also belongs to the scope of protection of the invention.

Claims (9)

1. a kind of image significance detection method based on low-rank Multiscale Fusion, it is characterised in that comprise the following steps：

Step 1, the image to being input into carry out single scale conspicuousness detection；

Step 2, to single scale conspicuousness detect after image carry out multiple dimensioned conspicuousness fusion treatment, obtain merge conspicuousness Figure；

Step 3, conspicuousness micronization processes are carried out to the fusion Saliency maps after multiple dimensioned conspicuousness fusion treatment, obtain final Collaboration Saliency maps picture.

2. a kind of image significance detection method based on low-rank Multiscale Fusion according to claim 1, its feature exists In：The specific processing method of the step 1 is comprised the following steps：

(1) image is too cut into multi-scale division figure and feature extraction is carried out；

(2) conspicuousness priori treatment is carried out using background transcendental method；

(3) conspicuousness calculating is carried out.

3. a kind of image significance detection method based on low-rank Multiscale Fusion according to claim 2, its feature exists In：Step method (1) is：By for be input into image, using SLIC methods will be input into image segmentation into super-pixel, And extract position feature, color characteristic and the textural characteristics of 122 dimensions.

4. a kind of image significance detection method based on low-rank Multiscale Fusion according to claim 2, its feature exists In：Step conspicuousness computational methods (3) are carried out using following conspicuousness model：

$SP\left(i\right)=\frac{\left|\right|\hat{S}(:,i)|{|}_2}{{\Σ}_i\left|\right|\hat{S}(:,i)|{|}_2}=\frac{\sqrt{{\Σ}_j{\left(\hat{S}\right(j,i\left)\right)}^2}}{{\Σ}_i\sqrt{{\Σ}_j{\left(\hat{S}\right(j,i\left)\right)}^2}}$

SP (i) is i-th significance value of super-pixel,It is the significance value of i-th super-pixel, j-th feature,It is i-th significance value vector of all features of super-pixel.

5. a kind of image significance detection method based on low-rank Multiscale Fusion according to claim 1, its feature exists In：The specific method of the step 2 is：First, piece image is divided into different yardsticks；Then, calculate on each yardstick Saliency maps；Finally, fusion conspicuousness is calculated by the way that the significance value of all yardsticks is multiplied by into corresponding adaptive weighting Figure.

6. a kind of image significance detection method based on low-rank Multiscale Fusion according to claim 5, its feature exists In：The adaptive weighting is expressed as follows：

Wherein, Z is a partition function；

The fusion Saliency maps are calculated using equation below：

$S_{map}^{fuse}=\Σ{\mathrm{\ω}}_i*S_{map}^i$

ω_iI-th adaptive weighting of the Saliency maps of yardstick is represented,I-th characteristic value of yardstick is represented,Represent the Saliency maps after multiple yardstick fusions.

7. a kind of image significance detection method based on low-rank Multiscale Fusion according to claim 1, its feature exists In：The processing method of the step 3 includes：

(1) smooth disposal is carried out to present image so that image reaches space smoothing；

(2) collaboration conspicuousness detection is carried out to image.

8. a kind of image significance detection method based on low-rank Multiscale Fusion according to claim 7, its feature exists In：(1) the step be to the method that present image carries out smooth disposal：Realized using following energy function：

$E={\Σ}_i{\mathrm{\ω}}_i^{bg}*s_i^2+{\Σ}_i{\mathrm{\ω}}_i^{fg}{(s_i-1)}^2+{\Σ}_{i,j\∈Nei\left(i\right)}{\mathrm{\ω}}_{ij}{(s_i-s_j)}^2$

Wherein, S_lThe significance value of each super-pixel i is represented,The probability of background is represented,The probability of expression prospect,Nei (i) represents i-th neighborhood of super-pixel, weights omega_ij：It is defined as：

Wherein,Represent the L2 distances of the color average in CIE-LAB color spaces, 6=10.

9. a kind of image significance detection method based on low-rank Multiscale Fusion according to claim 7, its feature exists In：(2) the step carries out collaboration conspicuousness detection to image and comprises the following steps：

1. single significant point detection：For a series of image I for giving_set={ I₁, I₂..., I_n, calculate each image Single Saliency maps, use S_iRepresent i-th single Saliency maps of image；

2. binary segmentation：Use adaptive threshold T_i：Single Saliency maps are divided into binary mask M_i, T_iIt is defined as：

T_i=α mean (S_i)

Wherein, α=2；

3. conspicuousness prior estimate is cooperateed with：GMM algorithms are built using 5 Gauss models for the foreground pixel in i-th picture Color model G_i, then with the mould M estimated in j-th picture_jProspect probability；N prospect probability is obtained for each picture Then every pictures are calculated collaboration conspicuousness priori to obtain the average value of these estimates by estimate；

4. collaboration conspicuousness is calculated：Collaboration conspicuousness priori is merged into single conspicuousness detection model and obtains last collaboration Saliency maps picture.