CN104077609A - Saliency detection method based on conditional random field - Google Patents

Abstract

The invention discloses a saliency detection method based on a conditional random field. Saliency detection is considered to be an image annotation problem. Multi-scale comparison is used, and salient feature graphs are obtained through three different saliency calculation modes including a center-periphery histogram and color and space distribution. The weight of saliency of each salient feature graph is calculated through CRF study, and a model parameter is obtained through a maximum likelihood estimation method to estimate the optimal solution. Finally, the CRF is used for detecting and testing images. The method can detect salient objects more precisely, results obtained through detection are high in resolution ratio, an object boarder is defined precisely, and the method is little in calculation complexity.

Description

A kind of conspicuousness detection method based on condition random field

Technical field

The invention belongs to image processes and image object detection field, particularly a kind of conspicuousness detection method based on condition random field.

Background technology

Vision is the most important sensory perceptions of the mankind, human brain receptible external information more than 90% come from the visually-perceptible of human eye.The major function of vision is exactly to explain the surrounding environment of people life, and with its generation information interaction, developing rapidly of infotech impels various image informations day by day to expand, these mass datas are processed and analyzed to people's computer system of having to.But it should be noted that: on the one hand, gathering way of view data wants fast more than the raising speed of computer process ability; On the other hand, the content that people are concerned about is a part very little in whole data acquisition conventionally.For this reason, all view data of the overall treatment of making no exception are unpractical, are also unnecessary.How, information useful and that merit attention important from whole data centralization finds and extraction is relevant to task part as soon as possible, i.e. vision significance test problems is exactly machine vision and the medium-term and long-term important problem facing of information processing research always.Saliency region detection technique taking vision attention as representative becomes one of important technology approach improving mass data screening real-time and precision of analysis.It is an important content during image is processed that conspicuousness detects, have a wide range of applications, and as the image based on conspicuousness is cut apart, image retrieval, image automatic cutting and the compression of image screen etc.

The essence that conspicuousness detects is a kind of visual attention model, and this model is the model of setting up according to vision noticing mechanism, and it can distribute limited information operation resource, makes perception possess selective power.Utilize vision noticing mechanism to obtain the signal portion the most easily arousing attention in image, and represent its significance with a width gray level image.Psychology of vision research discovery, human visual attention can be divided into two types: bottom-up data driven mode and top-down task-driven pattern.Bottom-up is at the visual processes initial stage, is not subject to experience and the impact of task at present, and it is salient region that the mankind exist particular concern region to scene.Top-down is the visual processes later stage, and the mankind, according to the experience of self and the target of task choosing concern, are familiar with target.

Condition random field (hereinafter referred: CRF) is a kind of discriminative model.Say that simply random field can regard the set of one group of stochastic variable as, after distributing to each position to give at random a value according to certain, it is all just called random field.Compared with production Hidden Markov Model (HMM) (hereinafter referred: HMM), CRF can select context dependent feature, does not have marking bias problem.CRF is a non-directed graph model, and this model is under the condition of the given observation sequence that needs mark, calculates the joint probability of whole flag sequence.CRF model mainly contains three basic tasks: 1. the selection of fundamental function, this is directly connected to model performance.2. parameter evaluation, from marking the parameter of training data study CRF model of mark, the i.e. weight vectors of each fundamental function.3. mode inference, under specified criteria random field models parameter, dopes most probable status switch.

Early stage its major defect of saliency algorithm is that resolution is low, and object boundary definition is poor, and computation complexity is high.

Summary of the invention

Goal of the invention: for the problem of prior art existence, the invention provides the conspicuousness detection method of the condition random field that a kind of resolution is high, computation complexity is high.

Summary of the invention: the invention provides a kind of conspicuousness detection method based on condition random field, comprise the steps:

Step 10: acquisition of image data;

Step 20, the image that step 10 is obtained carries out significant characteristics extraction with three kinds of diverse ways, obtains the significant characteristics figure that the significant characteristics function different from three kinds is corresponding;

Step 30: adopt the machine learning method of conditional random field models to train the image gathering in step 10, and obtain the optimal weights of the each significant characteristics figure obtaining in step 20;

Step 40: three kinds of different significant characteristics function partition function Z that step 20 is obtained are normalized;

Step 50: set up the condition random field models, three normalization significant characteristics function conditional random field models that step 40 is obtained combine;

Step 60: optimum solution is tried to achieve in combination step 50 being obtained by maximum-likelihood criterion, obtains optimized linear combination;

Step 70: a minimum rectangle frame for the conspicuousness pixel that step 60 is calculated, wherein minimum rectangle frame at least frame go out more than 95% conspicuousness pixel, obtain final result.

Further, in described step 20, described three kinds of methods of carrying out feature extraction are respectively: multiple dimensioned pairing comparision, central authorities histogram method and color space distribution around;

Wherein, described multiple dimensioned pairing comparision, comprises the following steps:

Step 211, based on down sample after Gaussian Blur, obtains the image of different resolution to the image collecting in step 10;

Step 212, in this pyramid diagram picture of six floor heights that step 211 is obtained, every layer of contrast linear combination obtains the significant characteristics figure that multiple dimensioned contrast characteristic's function is corresponding with it;

Described central authorities are histogram method around, comprises the following steps:

Step 221: mark the obvious object in the image that step 10 obtains with the rectangular area R of multiple different Aspect Ratios, construct the rectangular area R of surrounding that the area of multiple correspondences equates around multiple rectangular area R _s;

Step 222: on the image obtaining in step 221, calculate each remarkable rectangular area R centered by pixel x with around rectangular area R _sχ between RGB color histogram ²distance;

Step 223: the rectangular area R of more each different Aspect Ratios and around rectangular area R _sχ between RGB color histogram ²distance, now χ ²the maximum rectangular area R of distance is optimum rectangular area R ^*(x);

Step 224: central authorities-histogram feature function definition all optimums centered by the neighbor x ' rectangular area R around in step 221 around centered by neighbor x ' _s ^*gauss's weighting χ of (x ') ²apart from sum;

Described color space distribution, comprises the following steps:

Step 231: all colours in the image that step 10 is obtained represents with gauss hybrid models;

Step 232: the conditional probability of utilizing the calculation of parameter of model in step 231 to calculate each pixel to be assigned to a kind of color component;

Step 233: each color component in step 232 is calculated to corresponding level variance and vertical variance, obtain the space variance of tie element;

Step 234: color space distribution characteristics function definition is the space variance central authorities weighting sum that step 433 obtains.

Further, in step 50, described as follows by conditional random field models combination significant characteristics process:

Step 501, three normalization significant characteristics that step 40 is obtained, calculate respectively monobasic potential function F _k(a _x, I), F _k(a _x, I) and k notable feature of expression;

Step 502, three normalization significant characteristics that step 40 is obtained, calculate respectively binary potential function S (a _x, a _x', I) and pairing feature, wherein, binary potential function S (a _x, a _x', I) represent neighbor to be labeled as the value of the penalty term of different value, a _xrepresent the conspicuousness of x pixel, a _x'represent the conspicuousness of x neighbor pixel;

Step 503, the notable feature obtaining in the optimal weights of each significant characteristics figure that integrating step 30 obtains and step 501,502 and pairing feature, according to formula $P\left(A\right|I)=\frac{1}{Z}\exp (\underset{x}{\mathrm{\Σ}}\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{\mathrm{\λ}}_k{F}_k(a_x,I)+\underset{x,x^{\′}}{\mathrm{\Σ}}S(a_x,{a_x}^{\′},I))$ Carry out linear combination, wherein, A is for gathering label state set in image I, and Z is partition function, λ _krepresent the weight of k notable feature figure, K is the sum of notable feature figure, F _k(a _x, I) and be the potential function of unitary variant, F _k(a _x, I) and k notable feature figure of expression, S (a _x, a _x', I) be the mutual potential function of bivariate, S (a _x, a _x', I) expression neighbor x, the interaction relationship between x '.

Principle of work: the present invention detects conspicuousness to regard an image labeling problem as, uses multiple dimensioned contrast, central authorities-around histogram and color space this three kinds of different significances that distribute calculate notable feature figure.Learn the weight of the significance that calculates each notable feature figure by CRF, adopt maximum Likelihood to obtain model parameter estimation and obtain optimum solution.Finally utilize CRF to detect test pattern.

Beneficial effect: compared with prior art, method provided by the invention can more accurately detect well-marked target, detects the result resolution obtaining high, and precisely, method computation complexity is little in object boundary definition.

Brief description of the drawings

Fig. 1 is process flow diagram of the present invention;

Fig. 2 is that in the present invention, significant characteristics extracts process flow diagram;

Fig. 3 is the experiment comparison diagram of the present invention and prior art.

Embodiment

Below in conjunction with accompanying drawing, the present invention is described in detail.

As shown in Figure 1, a kind of conspicuousness detection method based on condition random field provided by the invention, comprises the steps:

Step 10: acquisition of image data; It is I that definition gathers image.

Step 20, the image that step 10 is obtained carries out significant characteristics extraction with three kinds of diverse ways, obtains the significant characteristics figure that the significant characteristics function different from three kinds is corresponding;

In the present embodiment, adopt multiple dimensioned pairing comparision, central histogram method around and color space distribution to carry out feature extraction.

1, multiple dimensioned pairing comparision

In conspicuousness detects, contrast method is to be the most often used on local feature.In the situation that not knowing remarkable object size, the conspicuousness that we adopt multi-scale method to carry out respectively regional area detects.Mainly comprise the following steps:

Step 211, based on down sample after Gaussian Blur, obtains the image of different resolution to the image collecting in step 10; The new figure image width and the height that obtain are 1/2 of original images simultaneously at every turn, and a series of images obtaining is called gaussian pyramid.

Step 212, in this pyramid diagram picture of six floor heights that step 211 is obtained, every layer of contrast linear combination obtains multiple dimensioned contrast characteristic's function the significant characteristics figure corresponding with it; Wherein, I ^lthe l tomographic image in pyramid, I ^l(x) be illustrated in the state value of pixel x on l tomographic image, i.e. I ^l(x) equal at 0 o'clock, represent that on l tomographic image, pixel x is conspicuousness pixel, I ^l(x) equal at 1 o'clock, being illustrated in pixel x on l tomographic image is non-conspicuousness pixel; I ^l(x ') represent pixel x on l tomographic image the state value of neighbor x ', the number of plies that pyramid has altogether is L=6, it is the window of 9x9 that N (x) represents.

2, central authorities' histogram method around

On given RGB color space basis, the pixel of adding up every kind of color component accounts for the ratio of the total pixel of image, thus obtain image shades of colour component ratio distribute be histogram.Suppose that an obvious object is gone out by rectangular area R frame, around it, we construct a rectangular area R with homalographic _s.Herein by calculating remarkable rectangular area R centered by pixel x and rectangular area R around it _s, their χ between RGB color histogram ²distance represents conspicuousness.Due to the target size difference of conspicuousness object, we select the rectangular area of different length breadth ratios to test.Mainly comprise the following steps:

Step 221: mark the obvious object in the image that step 10 obtains with the rectangular area R of five groups of different Aspect Ratios, construct the rectangular area R of surrounding that the area of multiple correspondences equates around multiple rectangular area R _s; Wherein, five groups of different Aspect Ratios are { 0.5,0.75,1.0,1.5,2};

Step 222: on the image obtaining in step 221, calculate each remarkable rectangular area R centered by pixel x with around rectangular area R _sχ between RGB color histogram ²distance;

Step 223: the rectangular area R of surrounding of the rectangular area R of more each different Aspect Ratios and rectangular area R homalographic _sχ between RGB color histogram ²distance, now χ ²the maximum rectangular area R of distance is optimum rectangular area R ^*(x);

Step 224: central authorities-histogram feature function definition all optimums centered by the neighbor x ' rectangular area R around in step 221 around centered by neighbor x ' _s ^*gauss's weighting χ of (x ') ²apart from sum; Central authorities-around histogram feature function is:

$f_h(x,I)\∝\underset{\left\{x^{\′}\right|x\∈R^{*}\left(x^{\′}\right)\}}{\mathrm{\Σ}}{w}_{{\mathrm{xx}}^{\′}}{\mathrm{\χ}}^2(R^{*}\left(x^{\′}\right),{R_s}^{*}\left(x^{\′}\right))$

Wherein weight w _{xx '}=exp (0.5 σ _{x '} ^-2|| x-x ' || ²), represent σ by Gauss's damping capacity _{x '} ²represent the covariance of pixel x', R ^*(x ') represents optimum rectangular area centered by neighbor x '.

3, color space distribution

Central authorities-around histogram has only been described a local characteristic area, but the distribution of the color space of the overall situation also can be described conspicuousness object in piece image, therefore combines part and overall feature herein and carries out conspicuousness detection.

Step 231: to step: all colours in 10 images that obtain represents with gauss hybrid models; The simplest method of space distribution of describing a color is exactly to calculate the space variance of color, adopts gauss hybrid models (GMM) here.All colours in image represents with GMM, and this model has three parameters wherein w _crepresent the weight of c color, u _crepresent the color average of c color, Σ _crepresent the covariance matrix of c color component, C represents the sum of color.

Step 232: the conditional probability of utilizing the calculation of parameter of model in step 231 to calculate each pixel to be assigned to a kind of color component; Conditional probability represents as follows:

$P\left(x\right|I_x)=\frac{w_{c}N\left(I_x\right|u_c,{\mathrm{\Σ}}_c)}{{\mathrm{\Σ}}_c{w}_{c}N\left(I_x\right|u_c,{\mathrm{\Σ}}_c)}$

Wherein, N (I _x| u _c, Σ _c) be a Gaussian distribution, for judging whether pixel x belongs to c color component.

Step 233: each color component in step 232 is calculated to corresponding level variance V _h(c) with vertical variance V _v(c), obtain the space variance of tie element | X| _c=V _v(c)+V _h(c);

Step 234: color space distribution characteristics function definition is the sum of the space variance central authorities weighting that obtains of step 433

$f_s(x,I)\∝\underset{c}{\mathrm{\Σ}}p\left(x\right|I_x)\·(1-V\left(c\right)).$

Step 30: set up the condition random field models, adopts the machine learning method of conditional random field models to train the image gathering in step 10, and obtain the optimal weights of the each significant characteristics figure obtaining in step 20; The optimal weights λ obtaining in the present embodiment ^*={ 0.25,0.48,0.27}

CRF definition: establishing G=(V, E) is a non-directed graph, and V is summit or node, represents stochastic variable.E is limit or arc, represents the dependence between stochastic variable.Y=(Y _v) _{v ∈ V}, Y is the stochastic variable Y taking node in G as index _vthe set forming.When under condition X, the pixel of the collection image I that observation sequence X represents here.Stochastic variable Y _vconditional probability distribution obey the Markov attribute of figure: p (Y _v| X, Y _w, w ≠ v)=p (Y _v| X, Y _w, w～v), wherein w～v represents that (w, v) is the limit of non-directed graph G, (X, Y) just forms a condition random field.The probability distribution of this model can be expressed as so:

$P\left(A\right|I)=\frac{1}{Z}\exp (\underset{x}{\mathrm{\Σ}}\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{\mathrm{\λ}}_k{F}_k(a_x,I)+\underset{x,x^{\′}}{\mathrm{\Σ}}S(a_x,{a_x}^{\′},I))$

Z is partition function, for the normalized of function.F _k(a _x, I) and be the potential function of unitary variant, F _k(a _x, I) and k notable feature of expression, λ _krepresent the weight of k notable feature.S (a _x, a _x', I) and be the mutual potential function of bivariate, S (a _x, a _x', I) represent neighbor to be labeled as the value of the penalty term of different value.

Step 40: three kinds of different significant characteristics function partition function Z that step 20 is obtained are normalized;

Three kinds of different significant characteristics define respectively a notable feature function f _x(x, I), is then normalized f _x(x, I) ∈ [0,1].Notable feature is defined as follows:

F _k(a _x,I)＝f _x(x,I) a _x＝0

Wherein, a _x=0 represents that x pixel is conspicuousness, a _x=1 represents that x pixel is non-conspicuousness.

The mutual potential function of bivariate is expressed as follows:

S(a _x,a _x',I)＝|a _x-a _x′|·exp(-βd _x,x′)

Wherein, d _{x, x '}=|| I _x-I _{x '}|| two norms that are colour-difference represent, β=(2<||I _x-I _{x '}|| ²>) ^-1represent color contrast weight parameter.

Step 50: three normalization significant characteristics function conditional random field models that step 40 is obtained combine;

Step 60: optimum solution is tried to achieve in combination step 50 being obtained by maximum-likelihood criterion, obtains optimized linear combination;

In order to obtain optimum characteristic line combination, to N width training image use maximal possibility estimation to train, wherein n represents n width training image, and it is got to expression formula after log is convex function, has optimum solution as follows:

${\stackrel{\&RightArrow;}{\mathrm{\&lambda;}}}^{*}=\arg \underset{\stackrel{\&RightArrow;}{\mathrm{\&lambda;}}}{\max }\underset{n}{\mathrm{\&Sigma;}}\log P\left(A^n\right|I^n;\stackrel{\&RightArrow;}{\mathrm{\&lambda;}})$

When obtaining after optimum parameter, by CRF model inference, try to achieve final optimum solution according to maximum posteriori criterion (MAP):

y ^*＝arg max P(A|I)

Step 70: step 60 is calculated to a series of state values, and wherein, 0 represents conspicuousness pixel, 1 represents non-conspicuousness pixel, go out conspicuousness pixel by a minimum rectangle circle, wherein minimum rectangle frame at least frame go out more than 95% conspicuousness pixel, obtain final result.

By multiple dimensioned pairing comparision obtained above, the optimal weights that the feature that around central authorities, histogram method and color space distribution obtain obtains by training carries out linear combination, according to confession

$y^{*}=\frac{1}{Z}\exp \left(\underset{x}{\mathrm{\&Sigma;}}(0.25f_c(x,I)+0.48f_h(x,I)0.27f_s(x,I))\right)+\underset{x,x^{\&prime;}}{\mathrm{\&Sigma;}}S(a_x,{a_x}^{\&prime;},I)$ Obtain the flag sequence of probability of occurrence maximum, thereby flag sequence is showed and can obtain final remarkable figure with gray level image.

By above-mentioned embodiment, visible tool of the present invention has the following advantages: in order to weigh the validity of algorithm herein, we can use assessment indicator: recall rate (Recall), accurate rate (Precision) and F-value (F-measure) compare.As shown in Figure 3, to adopt separately respectively local multiple dimensioned contrast, the assessment indicator that central authorities-around histogram and spatial color distribution feature obtain through test, and the assessment indicator that finally our method obtains three fundamental functions by optimum weights linear combination compares.In Fig. 3, adopting the experimental result of local multiple dimensioned contrast is separately 1, and central authorities-around histogrammic experimental result is 2 in employing separately, and adopting separately the experimental result that spatial color divides is 3, is 4 by three fundamental functions by the test findings of optimum weights linear combination.From Fig. 3 we can find out adopt the result that obtains of multiple dimensioned contrast characteristic be accurate rate very high time recall rate also very low, because obvious object interior zone is same item, so thereby inner contrast is not high has caused lower recall rate.Central authorities-around histogram obtains a good F-value in this several method, comprises some wrong pixels although this feature is described in background area, and this local feature can detect complete obvious object well.Spatial color distribution has lower accurate rate and the highest recall rate.In the conspicuousness of our research detects, the index of recall rate does not have accurate rate important.Three kinds of characteristic line combinations are obtained in this method, accurate rate, the index of recall rate and F-value is all higher.The present invention combines part and global characteristics, by CRF learning training parameter, three kinds of different characteristic line combinations is obtained to optimum result.Show by experiment, this method is with former method comparison, and this method can detect obvious object better more accurately.

Claims (3)

1. the conspicuousness detection method based on condition random field, is characterized in that, comprises the steps:

Step 10: acquisition of image data;

Step 20, the image that step 10 is obtained carries out significant characteristics extraction with three kinds of diverse ways, obtains the significant characteristics figure that the significant characteristics function different from three kinds is corresponding;

Step 30: set up random field models, adopt the machine learning method of conditional random field models to train the image gathering in step 10, and obtain the optimal weights of the each significant characteristics figure obtaining in step 20;

Step 40: three kinds of different significant characteristics function partition function Z that step 20 is obtained are normalized;

Step 50: set up the condition random field models, three normalization significant characteristics function conditional random field models that step 40 is obtained combine;

Step 60: optimum solution is tried to achieve in combination step 50 being obtained by maximum-likelihood criterion, obtains optimized linear combination;

Step 70: a minimum rectangle frame for the conspicuousness pixel that step 60 is calculated, wherein minimum rectangle frame at least frame go out more than 95% conspicuousness pixel, obtain final result.

2. a kind of conspicuousness detection method based on condition random field according to claim 1, it is characterized in that: in described step 20, described three kinds of methods of carrying out feature extraction are respectively: multiple dimensioned pairing comparision, central authorities histogram method and color space distribution around;

Wherein, described multiple dimensioned pairing comparision, comprises the following steps:

Step 211, based on down sample after Gaussian Blur, obtains the image of different resolution to the image collecting in step 10;

Step 212, in this pyramid diagram picture of six floor heights that step 211 is obtained, every layer of contrast linear combination obtains the significant characteristics figure that multiple dimensioned contrast characteristic's function is corresponding with it;

Described central authorities are histogram method around, comprises the following steps:

Step 221: mark the obvious object in the image that step 10 obtains with the rectangular area R of multiple different Aspect Ratios, construct the rectangular area R of surrounding that the area of multiple correspondences equates around multiple rectangular area R _s;

Step 222: on the image obtaining in step 221, calculate each remarkable rectangular area R centered by pixel x with around rectangular area R _sχ between RGB color histogram ²distance;

Step 223: the rectangular area R of more each different Aspect Ratios and around rectangular area R _sχ between RGB color histogram ²distance, now χ ²the maximum rectangular area R of distance is optimum rectangular area R ^*(x);

Step 224: central authorities-histogram feature function definition all optimums centered by the neighbor x ' rectangular area R around in step 221 around centered by neighbor x ' _s ^*gauss's weighting χ of (x ') ²apart from sum;

Described color space distribution, comprises the following steps:

Step 231: all colours in the image that step 10 is obtained represents with gauss hybrid models;

Step 232: the conditional probability of utilizing the calculation of parameter of model in step 231 to calculate each pixel to be assigned to a kind of color component;

Step 233: each color component in step 232 is calculated to corresponding level variance and vertical variance, obtain the space variance of tie element;

Step 234: color space distribution characteristics function definition is the sum of the space variance central authorities weighting that obtains of step 433.

3. a kind of conspicuousness detection method based on condition random field according to claim 1, is characterized in that: in step 50, described as follows by conditional random field models combination significant characteristics process:

Step 501, three normalization significant characteristics that step 40 is obtained, calculate respectively monobasic potential function F _k(a _x, I), F _k(a _x, I) and k notable feature of expression;

Step 502, three normalization significant characteristics that step 40 is obtained, calculate respectively binary potential function S (a _x, a _x', I) and pairing feature, wherein, binary potential function S (a _x, a _x', I) represent neighbor to be labeled as the value of the penalty term of different value, a _xrepresent the conspicuousness of x pixel, a _x'represent the conspicuousness of x neighbor pixel;

Step 503, the notable feature obtaining in the optimal weights of each significant characteristics figure that integrating step 30 obtains and step 501,502 and pairing feature, according to formula $P\left(A\right|I)=\frac{1}{Z}\exp (\underset{x}{\mathrm{\&Sigma;}}\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{\mathrm{\&lambda;}}_k{F}_k(a_x,I)+\underset{x,x^{\&prime;}}{\mathrm{\&Sigma;}}S(a_x,{a_x}^{\&prime;},I))$ Carry out linear combination, wherein, A is for gathering label state set in image I, and Z is partition function, λ _krepresent the weight of k notable feature figure, K is the sum of notable feature figure, F _k(a _x, I) and be the potential function of unitary variant, F _k(a _x, I) and k notable feature figure of expression, S (a _x, a _x', I) be the mutual potential function of bivariate, S (a _x, a _x', I) expression neighbor x, the interaction relationship between x '.