CN108846404B  Image significance detection method and device based on related constraint graph sorting  Google Patents
Image significance detection method and device based on related constraint graph sorting Download PDFInfo
 Publication number
 CN108846404B CN108846404B CN201810658629.3A CN201810658629A CN108846404B CN 108846404 B CN108846404 B CN 108846404B CN 201810658629 A CN201810658629 A CN 201810658629A CN 108846404 B CN108846404 B CN 108846404B
 Authority
 CN
 China
 Prior art keywords
 node
 value
 ith
 image
 formula
 Prior art date
 Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
 Active
Links
 238000001514 detection method Methods 0.000 title claims abstract description 39
 230000011218 segmentation Effects 0.000 claims abstract description 15
 239000011159 matrix material Substances 0.000 claims description 96
 238000004422 calculation algorithm Methods 0.000 claims description 38
 230000000875 corresponding Effects 0.000 claims description 31
 239000000126 substance Substances 0.000 claims description 17
 230000001276 controlling effect Effects 0.000 claims description 16
 238000004364 calculation method Methods 0.000 claims description 8
 238000010276 construction Methods 0.000 claims description 7
 238000010606 normalization Methods 0.000 claims description 7
 238000009795 derivation Methods 0.000 claims description 5
 238000011156 evaluation Methods 0.000 claims description 3
 230000036881 Clu Effects 0.000 description 8
 238000010586 diagram Methods 0.000 description 6
 230000000694 effects Effects 0.000 description 3
 238000000034 method Methods 0.000 description 3
 238000009499 grossing Methods 0.000 description 2
 230000000007 visual effect Effects 0.000 description 2
 241000668709 Dipterocarpus costatus Species 0.000 description 1
 238000004458 analytical method Methods 0.000 description 1
 238000004891 communication Methods 0.000 description 1
 230000002596 correlated Effects 0.000 description 1
 238000011161 development Methods 0.000 description 1
 230000018109 developmental process Effects 0.000 description 1
 238000005516 engineering process Methods 0.000 description 1
 239000012467 final product Substances 0.000 description 1
 230000004048 modification Effects 0.000 description 1
 238000006011 modification reaction Methods 0.000 description 1
 230000001537 neural Effects 0.000 description 1
 238000007781 preprocessing Methods 0.000 description 1
 238000005295 random walk Methods 0.000 description 1
 238000011160 research Methods 0.000 description 1
Classifications

 G—PHYSICS
 G06—COMPUTING; CALCULATING; COUNTING
 G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
 G06V10/00—Arrangements for image or video recognition or understanding
 G06V10/40—Extraction of image or video features
 G06V10/46—Descriptors for shape, contour or pointrelated descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
 G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
Abstract
The invention discloses an image significance detection method and device based on related constraint graph sorting, wherein the method comprises the following steps: performing superpixel segmentation on an image to be detected, establishing a closedloop graph model, and further calculating prior information of each superpixel node; extracting information such as color, texture, position and the like of an input image; obtaining a foreground probability value of each superpixel node; taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is larger than the second preset threshold; and calculating by using a model of related constraint graph sequencing to obtain the foreground probability S _ f of each superpixel node, and using the foreground probability value S _ f as a final significant estimation value S _ final. By applying the embodiment of the invention, the significance detection result can be more accurate.
Description
Technical Field
The invention relates to a method and a device for detecting saliency, in particular to a method and a device for detecting the saliency of an image based on related constraint graph sorting.
Background
With the rapid development of computer and network communication technologies, image data is increasing. The massive multimedia image data provides a great challenge for information processing, and a main research focus in recent years is how to efficiently store, analyze and process the image information. The saliency detection is used as an important preprocessing step for reducing the computational complexity in the field of computer vision, and the saliency target detection is used for positioning and segmenting the most salient foreground target from a scene. The application fields of the technology are particularly wide, such as: target detection and recognition, contentbased image retrieval, contextaware based image resizing, video target detection, and the like. How to quickly and accurately find the significant region of the image does not form a complete theoretical system, and has close relation with specific applications, and still remains a challenging subject for researchers.
Currently, visual information processing is generally performed using a bottomup approach. Bottomup methods are typically based on underlying visual information, so it is possible to efficiently detect detailed information of the image, rather than global shape information, and the detected salient region may contain only a portion of the object, or be easily mixed with the background. Many bottomup significance detection models have emerged in recent years: first, Itti et al proposed a neural networkbased saliency detection model that combines three feature channels in multiple scales to achieve fast scene analysis, and although the model was able to identify partially salient pixels, the results also included a large number of false positives. Harel et al propose a graphbased saliency detection method, which is a bottomup model, with final saliency results obtained by calculating dissimilarity. Chang et al constructed a graphical model combining semblance and regional saliency to obtain a better saliency estimate. Wang et al propose a saliency detection model that combines local graph structure and background prior, and an optimized framework, and the final experimental results have good performance in most scenarios. Jiang et al propose the use of an absorbing Markov chain model for saliency detection of images. Tu et al propose to use a minimum spanning tree model for saliency detection of images. Li et al propose to estimate significance values using a regularized random walk ranking model. Yang et al propose a significance detection algorithm based on manifold sorting of the figure, (hereinafter referred to as MR algorithm) this algorithm through screening out some foreground seed points and background seed points, then use the model of manifold sorting to calculate the correlation between these seed points and other nodes, thus get the final significance.
However, the MR algorithm is divided into two stages, the correlations between the remaining nodes and the obtained background seed points are first calculated, a preliminary significant result is obtained after negation is performed, then foreground seed points are obtained on the basis of the first stage, and then the correlations between the remaining nodes and the foreground seed points are calculated, so that a final result is obtained. The two sequencing processes of the method are independently performed, which can cause the technical problem of low accuracy of image significance detection.
Disclosure of Invention
The invention aims to provide an image significance detection method and device based on correlation constraint graph sorting, so as to solve the defects in the traditional manifold sorting model based on graphs.
The invention solves the technical problems through the following technical scheme:
the embodiment of the invention provides an image significance detection method based on related constraint graph sorting, which comprises the following steps:
a: aiming at each image to be detected, performing superpixel segmentation on the image to be detected by using a simple linear iterative clustering SLIC algorithm to obtain nonoverlapping superpixel blocks, then establishing a closedloop graph model by taking each nonoverlapping superpixel block as a node, and further calculating the central priori information of each node;
b: extracting information such as color, texture, position and the like of an input image;
c: acquiring a foreground probability value of each node by using an MR algorithm;
d: taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold;
e: and calculating by using a model of related constraint graph sequencing to obtain the foreground probability S _ f of each superpixel node, and using the foreground probability value S _ f as a final significant estimation value S _ final.
Optionally, the step a includes:
a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then, a nondirectional edge corresponding to each node is obtained, and a nondirectional graph model G is further constructed^{1}＝(V,E)；
A2: by means of the formula (I) and (II),central prior information is calculated for each node, wherein,
c_{i}central prior information of the ith node; x is the number of_{i}The abscissa of the central position of the ith node is; y is_{i}Is a longitudinal seat at the central position of the ith nodeMarking; (x)_{0},y_{0}) The coordinates of the center position of the whole image are shown; sigma_{1}The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.
Optionally, the undirected edge is obtained by:
for the neighboring node l of the two neighbors k of each node i, the formula is used, dist (k, l)   x_{k}x_{l}^{2}Calculating the Euclidean distance dist (k, l) between the color and k; if the Euclidean distance of the color is smaller than the threshold value theta, connecting a nondirectional edge between the node l and the node i, and continuing searching after finding the connected nodes until all the nodes are connected, wherein dist (k, l) is the Euclidean distance of the color between the ith node and the kth node; x is the number of_{k}Is the color value of the kth node; x is the number of_{l}Is the color value of the l node; and   is a modulo function.
Optionally, step B includes: according to the color, texture and position information of the extracted image; by means of the formula (I) and (II),calculating the weight of each undirected edge to construct a first incidence matrixWherein the content of the first and second substances,
a weight of an undirected edge between the ith node and the jth node; i and j are serial numbers of nodes, i is more than or equal to 0, and N is more than or equal to j; v. of_{i}Is a feature descriptor of the ith node, and v_{i}∈R^{65}，v_{i}＝[x_{i},y_{i},L_{i},a_{i},b_{i},c_{i},ω_{i}]；(x_{i},y_{i}) Representing the center position coordinates of each superpixel node; (L)_{i},a_{i},b_{i}) Expressed is the inclusion of each superpixel node in the CIE LAB color spaceThe color mean value of all contained pixel points; c. C_{i}Central prior information of the ith node; omega_{i}Is the LBP value of the ith node; v. of_{j}A feature descriptor for the jth node; sigma is a preset constant for controlling weight balance; n is the number of superpixel blocks.
Optionally, step C includes:
c1: acquiring the weight of each undirected edge of each node in the MR algorithm;
c2: constructing a second incidence matrix of the MR algorithm according to the weight of each nondirectional edgeWherein the content of the first and second substances,
is the weight of the edge between the ith and jth superpixels, andW^{2}is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. C_{i}The color mean value of all pixel points of the ith node in the CIE LAB color space; c. C_{j}The color mean value of all pixel points of the jth node in the CIELAB color space is obtained; sigma is a constant for controlling weight balance;
c3: according to the formula, D ═ diag { D ═ D_{11},…,d_{nn}calculating a degree matrix, wherein,
d is a degree matrix; diag { } is a diagonal matrix construction function; d_{ii}Is a degree matrix element, and the weight of the nondirectional edge corresponding to the second incidence matrix;
c4: for each node on the boundary, marking the marking value of the node according to boundary prior;
c5: using the formula, f: X → R^{m}Calculating the corresponding sorting weight of the image to be detected, wherein,
f is the ranking function, and f ═ f_{1},…,f_{n}]^{T}；f_{1}Is the ranking value of the 1 st node; f. of_{n}Is the ranking value of the nth node; n is the number of nodes; let y be [ y_{1},y_{2},…y_{n}]^{T}Representing a label vector, wherein the label value of the seed point is 1, and the label values of the rest nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; r^{m}Is mdimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;
c6: by using the formula of the ranking function,a closed solution is calculated in which, among other things,
f is a sorting function;solving a function minimum independent variable function; sigma is a summation function; f. of_{i}Is the ranking value of the ith node; f. of_{j}Is the ranking value of the jth node; y is_{i}Is the label value of the ith node;a weight of the undirected edge; d_{ii}The element of the ith row and the ith column in the degree matrix; d_{jj}The element of j row and j column in the degree matrix; mu is a balance parameter;
c7: using a formula in accordance with the closed solution,obtaining a nonnormalized solution, wherein,
d is a degree matrix; w^{2}Is a second incidence matrix; s is W^{2}The normalization matrix of (a);
c8: by means of the formula (I) and (II),respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain a background probability value f of each node under four conditions, wherein lambda is a preset parameter;
c9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation valuesThen carrying out negation to obtain a significant value of each node; and performing point multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value of the node.
Optionally, the step D includes:
d1: by means of the formula (I) and (II),acquiring a first preset threshold and a second preset threshold; wherein the content of the first and second substances,
h_{1}is a first preset threshold value; h is_{2}Is a second preset threshold; mean is an averaging function; max is a maximum evaluation function;
d2: by means of the formula (I) and (II),obtaining a foreground seed point set ind _ form and a background seed point set ind _ back, wherein,
ind _ form is a foreground seed point set; ind _ back is a background seed point set; theta is a preset parameter.
Optionally, the step E includes:
e1: using the formula, F: X → R^{n}Calculating the corresponding sorting weight of the image to be detected, wherein,
f is a sorting function; f_{i}Denoted is the rank value of the ith node, and F ═ F, g; f is the probability that each node belongs to the foreground, and g is the probability that each node belongs to the background;
e2: by means of the formula (I) and (II),acquiring a label value of each node; using the formula, Y ═ Y^{1},y^{2})∈R^{m×2}And obtaining a label vector of each node, wherein,
y is a label vector of each node; y is^{1}A label value for which the node belongs to the foreground; y is^{2}A label value for which the node belongs to a background; r^{m×2}Is a real space with 2m dimension;
e3: the model formula of the constructed related constraint graph ordering is as follows,
wherein the content of the first and second substances,
F^{*}is a closed solution; w_{ij}A first incidence matrix corresponding to the weight of the undirected edge between the ith node and the jth node; f_{i}Is the ranking value of the ith node; f_{j}Is the ranking value of the jth node; d_{i}Is a degree matrix element; f. of_{i}Foreground probability of the ith node; g_{i}The background probability of the ith node; w is a_{i}The characteristic weight of the ith node; x is the number of_{i}Is the characteristic of the ith node; b_{i}Is a bias parameter; beta is a_{1}Linear constraint coefficients to the foreground probability; beta is a_{2}Linear constraint coefficient to background probability;
e4: and (4) carrying out partial derivation on the foreground probability by using the sequencing model of the E3 step to obtain a significance value.
The embodiment of the invention also provides an image significance detection device based on the related constraint graph sorting, which comprises the following steps:
the first calculation module is used for carrying out superpixel segmentation on the image to be detected by using a simple linear iterative clustering SLIC algorithm aiming at each image to be detected to obtain nonoverlapping superpixel blocks, then establishing a closedloop graph model by taking each nonoverlapping superpixel block as a node, and further calculating the central prior information of each node;
the input module is used for extracting information such as color, texture, position and the like of an input image;
the second calculation module is used for acquiring the foreground probability value of each node by using the MR algorithm;
the first setting module is used for taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold;
and the second setting module is used for calculating the foreground probability S _ f and the background probability S _ g of each superpixel node by using the model ordered by the related constraint graph, and using the foreground probability value S _ f as a final significant estimation value S _ final.
Optionally, the first computing module is further configured to:
a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then, a nondirectional edge corresponding to each node is obtained, and a nondirectional graph model G is further constructed^{1}＝(V,E)；
A2: by means of the formula (I) and (II),central prior information is calculated for each node, wherein,
c_{i}central prior information of the ith node; x is the number of_{i}The abscissa of the central position of the ith node is; y is_{i}Is the longitudinal coordinate of the central position of the ith node; (x)_{0},y_{0}) The coordinates of the center position of the whole image are shown; sigma_{1}The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.
Optionally, the second calculating module is further configured to:
c1: acquiring the weight of each undirected edge of each node in the MR algorithm;
C2：constructing a second incidence matrix of the MR algorithm according to the weight of each nondirectional edgeWherein the content of the first and second substances,
is the weight of the edge between the ith and jth superpixels, andW^{2}is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. C_{i}The color mean value of all pixel points of the ith node in the CIE LAB color space; c. C_{j}The color mean value of all pixel points of the jth node in the CIELAB color space is obtained; sigma is a constant for controlling weight balance;
c3: according to the formula, D ═ diag { D ═ D_{11},…,d_{nn}calculating a degree matrix, wherein,
d is a degree matrix; diag { } is a diagonal matrix construction function; d_{ii}Is a degree matrix element, and weights of the nondirectional edges corresponding to the incidence matrix;
c4: for each node on the boundary, marking the marking value of the node according to boundary prior;
c5: using the formula, f: X → R^{m}Calculating the corresponding sorting weight of the image to be detected, wherein,
f is the ranking function, and f ═ f_{1},…,f_{n}]^{T}；f_{1}Is the ranking value of the ith node; f. of_{n}Is the ranking value of the nth node; n is the number of nodes; let y be [ y_{1},y_{2},…y_{n}]^{T}Representing the label vector, speciesThe label value of the child point is 1, and the label values of the other nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; r^{m}Is mdimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;
c6: by using the formula of the ranking function,a closed solution is calculated in which, among other things,
f is a sorting function;solving a function minimum independent variable function; sigma is a summation function; f. of_{i}Is the ranking value of the ith node; f. of_{j}Is the ranking value of the jth node; y is_{i}Is the label value of the ith node;a weight of the undirected edge; d_{ii}The element of the ith row and the ith column in the degree matrix; d_{jj}The element of j row and j column in the degree matrix; mu is a balance parameter;
c7: using a formula in accordance with the closed solution,obtaining a nonnormalized solution, wherein,
d is a degree matrix; w^{2}Is the second incidence matrix; s is W^{2}The normalization matrix of (a);
c8: by means of the formula (I) and (II),respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain a background probability value f of each node under four conditions, wherein lambda is a preset parameter;
c9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation valuesThen carrying out negation to obtain a significant value of each node; and performing point multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value of the node.
Optionally, the second calculating module is further configured to:
according to the color, texture and position information of the extracted image;
by means of the formula (I) and (II),calculating the weight of each undirected edge to construct a first incidence matrixWherein the content of the first and second substances,
a weight of an undirected edge between the ith node and the jth node; i and j are serial numbers of nodes, i is more than or equal to 0, and N is more than or equal to j; v. of_{i}Is a feature descriptor of the ith node, and v_{i}∈R^{65}，v_{i}＝[x_{i},y_{i}，L_{i},a_{i},b_{i},c_{i},ω_{i}]；(x_{i},y_{i}) Representing the center position coordinates of each superpixel node; (L)_{i},a_{i},b_{i}) The color mean value of all pixel points contained in the CIE LAB color space of each super pixel node is represented; c. C_{i}Central prior information of the ith node; omega_{i}Is the LBP value of the ith node; v. of_{j}A feature descriptor for the jth node; σ is a preset constant for controlling the weight balance.
Compared with the prior art, the invention has the following advantages:
by applying the embodiment of the invention, the association parameters between the foreground clues and the background clues are introduced when the graph ranking function is constructed, and compared with the prior art in which the association between the foreground clues and the background clues is not considered when the graph ranking function is constructed, more influence factors are considered, and the significance detection result is more accurate. Meanwhile, because the traditional mode based on composition ignores the effect of image features when the significance calculation is carried out, the final significance value is restrained by utilizing the linear learning of the image features, the composition information and the feature information are fully utilized, and the final detection result is further improved.
Drawings
Fig. 1 is a schematic flowchart of an image saliency detection method based on a related constraint map sorting according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a method for detecting image saliency based on relevance constraint map ranking according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a constructed closedloop graph model according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an image saliency detection apparatus based on a related constraint map sorting according to an embodiment of the present invention.
Detailed Description
The following examples are given for the detailed implementation and specific operation of the present invention, but the scope of the present invention is not limited to the following examples.
In order to solve the problems in the prior art, embodiments of the present invention provide a method and an apparatus for detecting image saliency based on a correlation constraint graph sorting.
Fig. 1 is a schematic flowchart of an image saliency detection method based on a related constraint map sorting according to an embodiment of the present invention; fig. 2 is a schematic diagram illustrating a principle of an image saliency detection method based on a related constraint map sorting according to an embodiment of the present invention, as shown in fig. 1 and fig. 2, the method includes:
s101: aiming at each image to be detected, performing superpixel segmentation on the image to be detected by using a simple linear iterative clustering SLIC algorithm to obtain nonoverlapping superpixel blocks, then establishing a closedloop graph model by using each nonoverlapping superpixel block as a node, and further calculating prior information of each node;
specifically, the step S101 may include: a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then obtaining the nondirectional edge corresponding to each node, wherein each node has an edge with the node directly adjacent to the node and the adjacent (two adjacent) node, calculating to obtain the connected edge aiming at the two adjacent nodes of each node by using a formula, and connecting the nodes on the four boundaries, thereby constructing the nondirectional graph model G^{1}(V, E); a2: by means of the formula (I) and (II),calculating central prior information of each node, wherein c_{i}Central prior information of the ith node; x is the number of_{i}The abscissa of the central position of the ith node is; y is_{i}Is the longitudinal coordinate of the central position of the ith node; (x)_{0},y_{0}) The coordinates of the center position of the whole image are shown; sigma_{1}The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.
Fig. 3 is a schematic structural diagram of a constructed closedloop graph model according to an embodiment of the present invention, and as shown in fig. 3, the construction process of fig. 3 is: carrying out superpixel segmentation on an image to be detected to obtain N superpixel blocks, taking each superpixel block as a node, connecting each superpixel with adjacent superpixels in a local region, constructing edges, and finally establishing a closedloop graph model G^{1}。
In practical application, the connection situation of the edge between each node can be divided into the following four situations:
1. there is an edge between each node i and its immediately adjacent node j.
2. Each node i has an edge with its two neighbors (neighbors) node k.
3. The neighboring node l of the two neighbors k of each node i uses the formula, dist (k, l)   x_{k}x_{l}^{2}Calculating the color Euclidean distance dist (k, l) between the node and the k, if the distance is smaller than a threshold value theta, considering that an edge is connected between the node and the node i, and continuing searching after finding the connected nodes until all the nodes are connected, wherein the dist (k, l) is the color Euclidean distance between the ith node and the kth node; x is the number of_{k}Is the color value of the kth node; x is the number of_{l}Is the color value of the l node; and   is a modulo function.
4. And all nodes positioned on the four boundaries are connected with each other to form closedloop connection around the image.
By applying the embodiment of the invention, compared with the closedloop graph model in the prior art, the 3 rd point is added into the constructed closedloop graph model, the local smooth range of each superpixel region is enlarged, the consistent characteristics among the superpixel regions in certain adjacent regions are better explained, and the accuracy of image significance detection is further improved.
S102: information such as color, texture, position, etc. of the input image is extracted.
Specifically, the color, texture, and position information of the extracted image may be used;
then the formula is utilized to obtain the final product,calculating the weight of each undirected edge to construct a first incidence matrixWherein the content of the first and second substances,
a weight of an undirected edge between the ith node and the jth node; w^{1}Is a first incidence matrix; i and j are serial numbers of nodes, i is more than or equal to 0, and N is more than or equal to j; v. of_{i}Being the ith nodeFeature descriptors, and v_{i}∈R^{65}，v_{i}＝[x_{i},y_{i},L_{i},a_{i},b_{i},c_{i},ω_{i}]；(x_{i},y_{i}) Representing the center position coordinates of each superpixel node; (L)_{i},a_{i},b_{i}) The color mean value of all pixel points contained in the CIE LAB color space of each super pixel node is represented; c. C_{i}Central prior information of the ith node; omega_{i}Is the LBP value of the ith node; v. of_{j}A feature descriptor for the jth node; sigma is a preset constant for controlling weight balance; n is the number of superpixel blocks.
As shown in fig. 2, in practical applications, the color feature of the image is a CIE LAB (LAB specified by the organization of the Commission International Eclairage light standard) color mean value that each superpixel region includes a pixel point; the texture features are Local Binary Patterns (LBP) features. Computing the weight W of an edge between two connected superpixel nodes by computing the difference in feature combinations between them^{2}。
S103: and acquiring the foreground probability value of each node by using an MR algorithm.
Specifically, the step S103 may include:
c1: performing superpixel segmentation on an image to be detected by using a SLIC algorithm, and obtaining n nonoverlapping superpixel block sets X ═ X by segmentation_{1},…x_{q},x_{q+1},…x_{n}wherein the first q superpixel blocks are marked query seed points, and the rest are unmarked superpixel nodes. Then, a closedloop graph model G is constructed^{2}(V, E). V represents all node sets, and E represents all undirected edge sets. Wherein, each node and its direct adjacent node, and adjacent node have the edge, the node on four borders interconnect.
C2: constructing a second incidence matrixWherein the content of the first and second substances,is the weight of the edge between the ith and jth superpixels, andW^{2}is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. C_{i}The color mean value of all pixel points of the ith node in the CIE LAB color space; c. C_{j}The color mean value of all pixel points of the jth node in the CIE LAB color space is obtained; σ is a constant that controls the balance of weights, and is usually positively correlated with the Euclidean distance of the color mean between two nodes.
C3: according to the formula, D ═ diag { D ═ D_{11},…,d_{nn}Calculating a degree matrix, wherein D is the degree matrix; diag { } is a diagonal matrix construction function; d_{ii}Is a degree matrix element, and and the weights of the nondirectional edges corresponding to the incidence matrix.
C4: for each node on the boundary, the labeled value of the node is labeled according to the boundary prior.
C5: using the formula, f: X → R^{m}And calculating the sorting weight corresponding to the image to be detected, wherein f is a sorting function, and f is [ f ═ f [ ]_{1},…,f_{n}]^{T}；f_{1}Is the ranking value of the 1 st node; f. of_{n}Is the ranking value of the nth node; n is the number of nodes; let y be [ y_{1},y_{2},…y_{n}]^{T}Representing a label vector, wherein the label value of the seed point is 1, and the label values of the rest nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; r^{m}Is mdimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;
c6: using a ranking functionThe formula is shown in the figure,computing a closed solution, wherein f^{*}Is a sorting function;solving a function minimum independent variable function; sigma is a summation function; f. of_{i}Is the ranking value of the ith node; f. of_{j}Is the ranking value of the jth node; y is_{1}Is the label value of the ith node;a weight of the undirected edge; d_{ii}The element of the ith row and the ith column in the degree matrix; d_{jj}The element of j row and j column in the degree matrix; μ is the equilibrium parameter.
C7: using a formula in accordance with the closed solution,obtaining a nonnormalized solution, wherein D is a degree matrix; w^{2}Is the second incidence matrix; s is W^{2}The normalization matrix of (2).
The closed solution calculated in the step C6 may be:
wherein, λ is a preset parameter; f. of^{*}Is the closed solution; i is the identity matrix.
From the closedsolution sum degree matrix, a nonnormalized solution can be obtained
C8: by means of the formula (I) and (II),respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain the background probability value f of each node under four conditions, wherein the background probability value f is the value of the background probability of each node under four conditionsAnd λ is a preset parameter.
In practical application, in image saliency detection, a degree matrix is usually used to replace an identity matrix in a closed solution, so as to obtain the above formula. For example, the correlation value between each node and the background seed points on the four boundaries, i.e., the background probability value of each node in the four cases, is calculated as f.
C9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation valuesThen carrying out negation to obtain a significant value of each node; and performing dot multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value.
In practical application, the value obtained by inverting the normalized background probability value is as follows:
the point multiplication of the significant values obtained in the four cases to obtain an initial result may be:
S_{bq}(i)＝S_{t}(i)×S_{b}(i)×S_{l}(i)×S_{r}(i)。
it is emphasized that in step S103, the foreground probability value of each node is obtained by using an MR algorithm, and S _ MR (foreground probability value) of the image to be detected, which is obtained in the first stage of the classical graphbased manifold sorting algorithm, is used.
Taking each superpixel as a node, connecting each superpixel with the superpixel which belongs to the adjacent superpixel in the local area, constructing edges, and then establishing a closedloop graph model G^{2}. Then extracting LAB color characteristics of each superpixel region, and calculating the weight W of the edge by calculating the difference value on the color characteristics between two connected superpixel nodes^{2}. The algorithm is mainly divided into two stages, and the supergrams on four boundaries of the image are selected according to the prior information calculated in the step S101The pixels are used as background Query points, then the correlation between each superpixel node and the background Query points is calculated by using a manifold sorting algorithm, the probability that each superpixel belongs to the background is calculated, then normalization is carried out, and then negation is carried out, so that an initial significant result is obtained. In the second stage, firstly, binarization is carried out on the initial result of the first stage, then foreground Query points are screened out, and then the correlation between each superpixel node and each foreground Query point is calculated by using a manifold sorting algorithm, so that the S _ MR (foreground probability value) of the image to be detected is obtained.
S104: taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold.
Specifically, the step S104 may include:
d1: by means of the formula (I) and (II),acquiring a first preset threshold and a second preset threshold; h is_{1}Is a first preset threshold value; h is_{2}Is a second preset threshold; mean is an averaging function; max is a maximum evaluation function;
d2: by means of the formula (I) and (II),acquiring a foreground seed point set ind _ form and a background seed point set ind _ back, wherein the ind _ form is the foreground seed point set; ind _ back is a background seed point set; theta is a preset parameter.
S105: and calculating by using a model of related constraint graph sequencing to obtain the foreground probability S _ f of each superpixel node, and using the foreground probability value S _ f as a final significant estimation value S _ final.
Specifically, the step S105 may include: e1: using the formula, F: X → R^{n}Calculating the corresponding sorting weight of the image to be detected, wherein F is a sorting function; f_{i}Indicating the ranking value of the ith nodeAnd F ═ F, g; f is the probability that each node belongs to the foreground, and g is the probability that each node belongs to the background; e2: by means of the formula (I) and (II),acquiring a label value of each node; using the formula, Y ═ Y^{1},y^{2})∈R^{m×2}And obtaining a label vector of each node, wherein,
y is a label vector of each node; y is^{1}A label value for which the node belongs to the foreground; y is^{2}A label value for which the node belongs to a background; r^{m×2}Is a real space of 2m dimensions.
E3: the model formula of the constructed related constraint graph ordering is as follows,
wherein, F^{*}Is a closed solution; w_{ij}A first incidence matrix corresponding to the weight of the undirected edge between the ith node and the jth node; f_{i}Is the ranking value of the ith node; f_{j}Is the ranking value of the jth node; d_{i}Is a degree matrix element; f. of_{i}Foreground probability of the ith node; g_{i}The background probability of the ith node; w is a_{i}The characteristic weight of the ith node; x is the number of_{i}Is the characteristic of the ith node; b_{i}Is a bias parameter; beta is a_{1}Linear constraint coefficients to the foreground probability; beta is a_{2}Linear constraint coefficient to background probability; e4: and (4) carrying out partial derivation on the foreground probability by using the sequencing model of the E3 step to obtain a significance value.
In the formula in step E3, the first polynomial is a smoothing term, and since the surrounding area of the area with a certain characteristic in the image also has similar features to it, i.e. the scores of the rankings between the nodes in the adjacent local areas are considered as similar as possible, the smoothing term can be added; the second term is a fitting term, so that the difference between the ranking value we finally calculate and the initial label value we give is as small as possible; the third term is a constraint term for f and g, in order to make the correlation between f and g calculated as small as possible; the fourth and fifth terms are linear constraint terms for f and g respectively, and the final significant value is constrained by linear learning of image features.
In practical applications, the step of E4 may include:
1) simplifying the model formula of the constructed related constraint graph sequencing to obtain an optimized formula,
2) fixing f can obtain the optimal solution of b and W, so there are:
and
wherein the content of the first and second substances,1 is a vector of all 1's. I ∈ R is the identity matrix, so one can get
Wherein the content of the first and second substances,
3) writing the solving problem of the model with the sequencing of the related constraint graphs into the following formula:
J＝Tr[F^{T}A^{*}FμF^{T}Y]+λf^{T}g+β_{1}X^{T}W_{f}+b_{f}1f^{2}+β_{2}X^{T}W_{g}+b_{g}1g^{2}wherein, in the step (A),
A^{*}＝(1+μ)DW。
4) replacing F in the formula with FIn the alternative, F is (F, g), and Y is (Y)_{1}，y_{2}) And solving the reduction can obtain a formula,
5) and f in the step 4) is subjected to derivation to obtain the following results:
6) and g in the step 4) is subjected to derivation to obtain the following results:
7) from 5) and 6) can be obtained:
8) calculated according to the formula of 7):
f^{*}＝μ(λ^{2}I4(A^{*})^{2}2A^{*}β_{1}B2β_{2}BA^{*}β_{2}β_{1}B^{2})^{1}(λy_{2}2A^{*}y_{1}β_{2}By_{1})；
will f is^{*}As the final significant estimate S final.
By applying the embodiment shown in the figure 1 of the invention, when the graph ranking function is constructed, the association parameters between the foreground clues and the background clues are introduced, and when the association between each superpixel node and the given foreground query point and the background query is calculated at the same time, a correlation constraint condition is added, so that the correlation between the obtained foreground probability value and the obtained background probability value is reduced. Meanwhile, because the traditional mode based on composition ignores the effect of image features when the significance calculation is carried out, the final significance value is restrained by utilizing the linear learning of the image features, the composition information and the feature information are fully utilized, and the final detection result is further improved.
Corresponding to the image significance detection method based on the related constraint graph sorting provided by the embodiment of the invention, the embodiment of the invention also provides an image significance detection device based on the related constraint graph sorting.
Fig. 4 is a schematic structural diagram of an image saliency detection apparatus based on a related constraint map sorting according to an embodiment of the present invention, as shown in fig. 4, the apparatus includes:
the first calculation module 401 is configured to perform superpixel segmentation on each image to be detected by using a simple linear iterative clustering SLIC algorithm to obtain nonoverlapping superpixel blocks, then establish a closedloop graph model by using each nonoverlapping superpixel block as a node, and further calculate prior information of each node;
an input module 402, configured to extract information such as color, texture, and position of an input image;
a second calculating module 403, configured to obtain a foreground probability value of each node by using an MR algorithm;
a first setting module 404, configured to use a set of nodes with a foreground probability value greater than a first preset threshold as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold;
and a second setting module 405, configured to calculate a foreground probability S _ f and a background probability S _ g of each superpixel node by using the model ordered by the relevant constraint graph, and use the foreground probability value S _ f as a final significant estimation value S _ final.
By applying the embodiment shown in fig. 4 of the invention, when the graph ranking function is constructed, the association parameters between the foreground clues and the background clues are introduced, and when the association between each superpixel node and the given foreground query point and the background query is calculated at the same time, a correlation constraint condition is added, so that the correlation between the obtained foreground probability value and the obtained background probability value is reduced. Meanwhile, because the traditional mode based on composition ignores the effect of image features when the significance calculation is carried out, the final significance value is restrained by utilizing the linear learning of the image features, the composition information and the feature information are fully utilized, and the final detection result is further improved.
In a specific implementation manner of the embodiment of the present invention, the first calculating module 404 is further configured to:
a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then, a nondirectional edge corresponding to each node is obtained, and a nondirectional graph model G is further constructed^{1}＝(V,E)；
A2: by means of the formula (I) and (II),central prior information is calculated for each node, wherein,
c_{i}central prior information of the ith node; x is the number of_{i}The abscissa of the central position of the ith node is; y is_{i}Is the longitudinal coordinate of the central position of the ith node; (x)_{0},y_{0}) The coordinates of the center position of the whole image are shown; sigma_{1}The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.
In a specific implementation manner of the embodiment of the present invention, the second calculating module 403 is further configured to:
c1: acquiring the weight of each undirected edge of each node in the MR algorithm;
c2: constructing a second incidence matrix of the MR algorithm according to the weight of each nondirectional edgeWherein the content of the first and second substances,
is the weight of the edge between the ith and jth superpixels, andW^{2}is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. C_{i}The color mean value of all pixel points of the ith node in the CIE LAB color space; c. C_{j}The color mean value of all pixel points of the jth node in the CIELAB color space is obtained; sigma is a constant for controlling weight balance;
c3: according to the formula, D ═ diag { D ═ D_{11},…,d_{nn}calculating a degree matrix, wherein,
d is a degree matrix; diag { } is a diagonal matrix construction function; d_{ii}Is a degree matrix element, and weights of the nondirectional edges corresponding to the incidence matrix;
c4: for each node on the boundary, marking the marking value of the node according to boundary prior;
c5: using the formula, f: X → R^{m}Calculating the corresponding sorting weight of the image to be detected, wherein,
f is the ranking function, and f ═ f_{1},…,f_{n}]^{T}；f_{1}Is the ranking value of the 1 st node; f. of_{n}Is the ranking value of the nth node; n is the number of nodes; let y be [ y_{1}，y_{2}，…y_{n}]^{T}Representing a label vector, wherein the label value of the seed point is 1, and the label values of the rest nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; r^{m}Is mdimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;
c6: by using the formula of the ranking function,a closed solution is calculated in which, among other things,
f^{*}is a sorting function;solving a function minimum independent variable function; sigma is a summation function; f. of_{i}Is the ranking value of the ith node; f. of_{j}Is the ranking value of the jth node; y is_{i}Is the label value of the ith node;a weight of the undirected edge; d_{ii}The element of the ith row and the ith column in the degree matrix; d_{jj}The element of j row and j column in the degree matrix; mu is a balance parameter;
c7: using a formula in accordance with the closed solution,obtaining a nonnormalized solution, wherein D is a degree matrix; w^{2}Is the second incidence matrix; s is W^{2}The normalization matrix of (a);
c8: by means of the formula (I) and (II),respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain the background probability value f of each node under four conditions, wherein lambda is a preset parameter；
C9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation valuesThen carrying out negation to obtain a significant value of each node; and performing point multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value of the node.
In a specific implementation manner of the embodiment of the present invention, the second calculating module 403 is further configured to:
by means of the formula (I) and (II),calculating the weight of each undirected edge to construct a first incidence matrixWherein the content of the first and second substances,
a weight of an undirected edge between the ith node and the jth node; i and j are serial numbers of nodes, i is more than or equal to 0, and N is more than or equal to j; v. of_{i}Is a feature descriptor of the ith node, and v_{i}∈R^{65}，v_{i}＝[x_{i},y_{i},L_{i},a_{i},b_{i},c_{i},ω_{i}]；(x_{i},y_{i}) Representing the center position coordinates of each superpixel node; (L)_{i},a_{i},b_{i}) The color mean value of all pixel points contained in the CIE LAB color space of each super pixel node is represented; c. C_{i}Central prior information of the ith node; omega_{i}Is the LBP value of the ith node; v. of_{j}A feature descriptor for the jth node; σ is a preset constant for controlling the weight balance.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (8)
1. An image saliency detection method based on relevance constraint graph sorting is characterized by comprising the following steps:
a: aiming at each image to be detected, performing superpixel segmentation on the image to be detected by using a simple linear iterative clustering SLIC algorithm to obtain nonoverlapping superpixel blocks, then establishing a closedloop graph model by taking each nonoverlapping superpixel block as a node, and further calculating the central priori information of each node;
b: extracting color, texture and position information of an input image;
c: obtaining a foreground probability value of each node by using a significance detection algorithm based on manifold sorting of a graph;
d: taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold;
e: calculating by using a model of related constraint graph sequencing to obtain a foreground probability S _ f of each super pixel node, and using the foreground probability value S _ f as a final significant estimation value S _ final;
and step C, comprising:
c1: acquiring the weight of each undirected edge of each node in a significance detection algorithm based on manifold sorting of a graph;
c2: constructing a second incidence matrix of the significance detection algorithm based on manifold sorting of the graph according to the weight of each undirected edgeWherein the content of the first and second substances,
is the weight of the edge between the ith super pixel and the jth super pixel,and isW^{2}Is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. C_{i}The color mean value of all pixel points of the ith node in the CIE LAB color space; c. C_{j}The color mean value of all pixel points of the jth node in the CIE LAB color space is obtained; sigma is a constant for controlling weight balance;
c3: according to the formula, D ═ diag { D ═ D_{11},…,d_{nn}calculating a degree matrix, wherein,
d is a degree matrix; diag { } is a diagonal matrix construction function; d_{ii}Is a degree matrix element, and weights of the nondirectional edges corresponding to the incidence matrix;
c4: for each node on the boundary, marking the marking value of the node according to boundary prior;
c5: using the formula, f: X → R^{m}Calculating the corresponding sorting weight of the image to be detected, wherein,
f is the ranking function, and f ═ f_{1},…,f_{n}]^{T}；f_{1}Is the ranking value of the 1 st node; f. of_{n}Is the ranking value of the nth node; n is the number of nodes; let y be [ y_{1},y_{2},…y_{n}]^{T}Representing a label vector, wherein the label value of the seed point is 1, and the label values of the rest nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; r^{m}Is mdimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;
c6: by using the formula of the ranking function,a closed solution is calculated in which, among other things,
f^{*}is a sorting function;solving a function minimum independent variable function; sigma is a summation function; f. of_{i}Is the ranking value of the ith node; f. of_{j}Is the ranking value of the jth node; y is_{i}Is the label value of the ith node;a weight of the undirected edge; d_{ii}The element of the ith row and the ith column in the degree matrix; d_{jj}The element of j row and j column in the degree matrix; mu is a balance parameter;
c7: using a formula in accordance with the closed solution,obtaining a nonnormalized solution, wherein,
d is a degree matrix; w^{2}Is the second incidence matrix; s is W^{2}The normalization matrix of (a);
c8: by means of the formula (I) and (II),respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain a background probability value f of each node under four conditions, wherein lambda is a preset parameter;
c9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation valuesThen carrying out negation to obtain a significant value of each node; and performing point multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value of the node.
2. The method for detecting the significance of the image based on the related constraint map sorting as claimed in claim 1, wherein the step A comprises:
a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then, a nondirectional edge corresponding to each node is obtained, and a nondirectional graph model G is further constructed^{1}＝(V,E)；
A2: by means of the formula (I) and (II),central prior information is calculated for each node, wherein,
c_{i}central prior information of the ith node; x is the number of_{i}The abscissa of the central position of the ith node is; y is_{i}Is the longitudinal coordinate of the central position of the ith node; (x)_{0},y_{0}) The coordinates of the center position of the whole image are shown; sigma_{1}The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.
3. The method for detecting the saliency of images based on the ordering of related constraint maps according to claim 2, characterized in that the undirected edges are obtained by:
for the neighboring node l of the two neighbors k of each node i, the formula is used, dist (k, l)   x_{k}x_{l}^{2}Calculating the Euclidean distance between the color of the color k and the color k; if the Euclidean distance of the color is smaller than the threshold value theta, connecting a nondirectional edge between the node l and the node i, and continuing searching after finding the connected nodes until all the nodes are connected, wherein dist (k, l) is the Euclidean distance of the color between the ith node and the kth node; x is the number of_{k}Is the color value of the kth node; x is the number of_{l}Is the color value of the l node; and   is a modulo function.
4. The method for detecting the saliency of images based on the ordering of correlation constraint maps according to claim 1, wherein the step B comprises:
according to the color, texture and position information of the extracted image; by means of the formula (I) and (II),calculating the weight of each undirected edge to construct a first incidence matrixWherein the content of the first and second substances,
a weight of an undirected edge between the ith node and the jth node; w^{1}Is a first incidence matrix; i and j are serial numbers of nodes, i is more than or equal to 0, and N is more than or equal to j; v. of_{i}Is a feature descriptor of the ith node, and v_{i}∈R^{65}，v_{i}＝[x_{i},y_{i},L_{i},a_{i},b_{i},c_{i},ω_{i}]；(x_{i},y_{i}) Representing the center position coordinates of each superpixel node; (L)_{i},a_{i},b_{i}) The color mean value of all pixel points contained in the CIE LAB color space of each super pixel node is represented; c. C_{i}Central prior information of the ith node; omega_{i}Is the LBP value of the ith node; v. of_{j}A feature descriptor for the jth node; sigma is a preset constant for controlling weight balance; n is the number of superpixel blocks.
5. The method for detecting the significance of the image based on the related constraint map sorting as claimed in claim 1, wherein the step D comprises:
d1: by means of the formula (I) and (II),acquiring a first preset threshold and a second preset threshold; wherein the content of the first and second substances,
h_{1}is a first preset threshold value;h_{2}is a second preset threshold; mean is an averaging function; max is a maximum evaluation function;
d2: by means of the formula (I) and (II),obtaining a foreground seed point set ind _ form and a background seed point set ind _ back, wherein,
ind _ form is a foreground seed point set; ind _ back is a background seed point set; theta is a preset parameter.
6. The method for detecting the significance of the image based on the related constraint map ordering according to claim 1, wherein the step E comprises the following steps:
e1: using the formula, F: X → R^{n}Calculating the corresponding sorting weight of the image to be detected, wherein,
f is a sorting function; f_{i}Denoted is the rank value of the ith node, and F ═ F, g; f is the probability that each node belongs to the foreground, and g is the probability that each node belongs to the background;
e2: by means of the formula (I) and (II),acquiring a label value of each node; using the formula, Y ═ Y^{1},y^{2})∈R^{m×2}And obtaining a label vector of each node, wherein,
y is a label vector of each node; y is^{1}A label value for which the node belongs to the foreground; y is^{2}A label value for which the node belongs to a background; r^{m×2}Is a real space with 2m dimension;
e3: the model formula of the constructed related constraint graph ordering is as follows,
wherein the content of the first and second substances,
F^{*}is a closed solution; w_{ij}As a nondirectional edge between the ith node and the jth nodeA first incidence matrix corresponding to the weight of (a); f_{i}Is the ranking value of the ith node; f_{j}Is the ranking value of the jth node; d_{i}Is a degree matrix element; f. of_{i}Foreground probability of the ith node; g_{i}The background probability of the ith node; w is a_{i}The characteristic weight of the ith node; x is the number of_{i}Is the characteristic of the ith node; b_{i}Is a bias parameter; beta is a_{1}Linear constraint coefficients to the foreground probability; beta is a_{2}Linear constraint coefficient to background probability;
e4: and (4) carrying out partial derivation on the foreground probability by using the sequencing model of the E3 step to obtain a significance value.
7. An apparatus for detecting image saliency based on relevance constraint map ordering, the apparatus comprising:
the first calculation module is used for carrying out superpixel segmentation on the image to be detected by using a simple linear iterative clustering SLIC algorithm aiming at each image to be detected to obtain nonoverlapping superpixel blocks, then establishing a closedloop graph model by taking each nonoverlapping superpixel block as a node, and further calculating the central prior information of each node;
the input module is used for extracting color, texture and position information of an input image;
the second calculation module is used for acquiring the foreground probability value of each node by utilizing a significance detection algorithm based on manifold sorting of the graph;
the first setting module is used for taking a set of nodes with foreground probability values larger than a first preset threshold value as a foreground seed point set ind _ form; taking a set of nodes with the foreground probability value smaller than a second preset threshold value as a background seed point set ind _ back; the first preset threshold is greater than the second preset threshold;
the second setting module is used for calculating a foreground probability S _ f and a background probability S _ g of each superpixel node by using a model ordered by a related constraint graph, and using the foreground probability value S _ f as a final significant estimation value S _ final;
the second computing module is further configured to:
c1: acquiring the weight of each undirected edge of each node in a significance detection algorithm based on manifold sorting of a graph;
c2: constructing a second incidence matrix of the significance detection algorithm based on manifold sorting of the graph according to the weight of each undirected edgeWherein the content of the first and second substances,
is the weight of the edge between the ith and jth superpixels, andW^{2}is a second incidence matrix; i, j belongs to V, and i is the serial number of the ith node; j is the serial number of the jth node; c. C_{i}The color mean value of all pixel points of the ith node in the CIE LAB color space; c. C_{j}The color mean value of all pixel points of the jth node in the CIE LAB color space is obtained; sigma is a constant for controlling weight balance;
c3: according to the formula, D ═ diag { D ═ D_{11},…,d_{nn}calculating a degree matrix, wherein,
d is a degree matrix; diag { } is a diagonal matrix construction function; d_{ii}Is a degree matrix element, and the weight of the nondirectional edge corresponding to the second incidence matrix;
c4: for each node on the boundary, marking the marking value of the node according to boundary prior;
c5: using the formula, f: X → R^{m}Calculating the corresponding sorting weight of the image to be detected, wherein,
f is the ranking function, and f ═ f_{1},…,f_{n}]^{T}；f_{1}Is the ranking value of the 1 st node; f. of_{n}Is the ranking value of the nth node; n is the number of nodes; let y be [ y_{1},y_{2},…y_{n}]^{T}Representing a label vector, wherein the label value of the seed point is 1, and the label values of the rest nodes are 0; x is a characteristic matrix corresponding to the input image; r is real number space; r^{m}Is mdimensional real number space; m is a spatial dimension; y is a vector formed by label values of all the seed nodes;
c6: by using the formula of the ranking function,a closed solution is calculated in which, among other things,
f^{*}is a sorting function;solving a function minimum independent variable function; sigma is a summation function; f. of_{i}Is the ranking value of the ith node; f. of_{j}Is the ranking value of the jth node; y is_{i}Is the label value of the ith node; y is_{n}The weight of the nth node;a weight of the undirected edge; d_{ii}The element of the ith row and the ith column in the degree matrix; d_{jj}The element of j row and j column in the degree matrix; mu is a balance parameter;
c7: using a formula in accordance with the closed solution,obtaining a nonnormalized solution, wherein,
d is a degree matrix; w^{2}Is the second incidence matrix; s is W^{2}The normalization matrix of (a);
c8: by means of the formula (I) and (II),respectively calculating the correlation between each node and the background seed points on the four boundaries to obtain a background probability value f of each node under four conditions, wherein lambda is a preset parameter;
c9: normalizing correlation values between each node and background seed points on four boundaries to obtain correlation valuesThen carrying out negation to obtain a significant value of each node; and performing point multiplication on the significant values obtained under the four conditions to obtain an initial result S _ MR as a foreground probability value of the node.
8. The apparatus according to claim 7, wherein the first computing module is further configured to:
a1: for each image to be detected, performing superpixel segmentation on the image into N superpixel blocks by using a SLIC algorithm, wherein each superpixel is used as a node in a set V; then, a nondirectional edge corresponding to each node is obtained, and a nondirectional graph model G is further constructed^{1}＝(V,E)；
A2: by means of the formula (I) and (II),central prior information is calculated for each node, wherein,
c_{i}central prior information of the ith node; x is the number of_{i}The abscissa of the central position of the ith node is; y is_{i}Is the longitudinal coordinate of the central position of the ith node; (x)_{0},y_{0}) The coordinates of the center position of the whole image are shown; sigma_{1}The balance parameter is used for controlling the discrete degree of the calculated position distance; exp () is an exponential function with a natural base number as the base number; i is the number of nodes.
Priority Applications (1)
Application Number  Priority Date  Filing Date  Title 

CN201810658629.3A CN108846404B (en)  20180625  20180625  Image significance detection method and device based on related constraint graph sorting 
Applications Claiming Priority (1)
Application Number  Priority Date  Filing Date  Title 

CN201810658629.3A CN108846404B (en)  20180625  20180625  Image significance detection method and device based on related constraint graph sorting 
Publications (2)
Publication Number  Publication Date 

CN108846404A CN108846404A (en)  20181120 
CN108846404B true CN108846404B (en)  20211001 
Family
ID=64203559
Family Applications (1)
Application Number  Title  Priority Date  Filing Date 

CN201810658629.3A Active CN108846404B (en)  20180625  20180625  Image significance detection method and device based on related constraint graph sorting 
Country Status (1)
Country  Link 

CN (1)  CN108846404B (en) 
Families Citing this family (4)
Publication number  Priority date  Publication date  Assignee  Title 

CN110111353B (en) *  20190429  20200124  河海大学  Image significance detection method based on Markov background and foreground absorption chain 
CN110188763B (en) *  20190528  20210430  江南大学  Image significance detection method based on improved graph model 
CN110287802A (en) *  20190529  20190927  南京邮电大学  Human eye fixation point prediction technique based on optimization display foreground and background seed 
CN110298842A (en) *  20190610  20191001  上海工程技术大学  A kind of rail clip image position method based on superpixel node sequencing 
Citations (3)
Publication number  Priority date  Publication date  Assignee  Title 

US6038344A (en) *  19960712  20000314  The United States Of America As Represented By The Secretary Of The Navy  Intelligent hypersensor processing system (IHPS) 
CN104123734A (en) *  20140722  20141029  西北工业大学  Visible light and infrared detection result integration based moving target detection method 
CN104715251A (en) *  20150213  20150617  河南科技大学  Salient object detection method based on histogram linear fitting 

2018
 20180625 CN CN201810658629.3A patent/CN108846404B/en active Active
Patent Citations (3)
Publication number  Priority date  Publication date  Assignee  Title 

US6038344A (en) *  19960712  20000314  The United States Of America As Represented By The Secretary Of The Navy  Intelligent hypersensor processing system (IHPS) 
CN104123734A (en) *  20140722  20141029  西北工业大学  Visible light and infrared detection result integration based moving target detection method 
CN104715251A (en) *  20150213  20150617  河南科技大学  Salient object detection method based on histogram linear fitting 
Also Published As
Publication number  Publication date 

CN108846404A (en)  20181120 
Similar Documents
Publication  Publication Date  Title 

CN108846404B (en)  Image significance detection method and device based on related constraint graph sorting  
Li et al.  Robust saliency detection via regularized random walks ranking  
Liu et al.  Adaptive partial differential equation learning for visual saliency detection  
CN107103326B (en)  Collaborative significance detection method based on superpixel clustering  
CN107066559B (en)  Threedimensional model retrieval method based on deep learning  
Li et al.  SAR image change detection using PCANet guided by saliency detection  
CN112184752A (en)  Video target tracking method based on pyramid convolution  
CN111881714A (en)  Unsupervised crossdomain pedestrian reidentification method  
US20180232400A1 (en)  Sketchbased image searching system using cellorientation histograms and outline extraction based on mediumlevel features  
CN111126385A (en)  Deep learning intelligent identification method for deformable living body small target  
CN109086777B (en)  Saliency map refining method based on global pixel characteristics  
CN111241924A (en)  Face detection and alignment method and device based on scale estimation and storage medium  
CN110598715A (en)  Image recognition method and device, computer equipment and readable storage medium  
CN107862680B (en)  Target tracking optimization method based on correlation filter  
CN111914642A (en)  Pedestrian reidentification method, device, equipment and medium  
CN110135435B (en)  Saliency detection method and device based on breadth learning system  
CN110827312A (en)  Learning method based on cooperative visual attention neural network  
CN110765882A (en)  Video tag determination method, device, server and storage medium  
Lee et al.  Propertyspecific aesthetic assessment with unsupervised aesthetic property discovery  
CN112529005A (en)  Target detection method based on semantic feature consistency supervision pyramid network  
CN112712546A (en)  Target tracking method based on twin neural network  
CN110472092B (en)  Geographical positioning method and system of street view picture  
CN112101150A (en)  Multifeature fusion pedestrian reidentification method based on orientation constraint  
CN111768415A (en)  Image instance segmentation method without quantization pooling  
Duan et al.  Bioinspired visual attention model and saliency guided object segmentation 
Legal Events
Date  Code  Title  Description 

PB01  Publication  
PB01  Publication  
SE01  Entry into force of request for substantive examination  
SE01  Entry into force of request for substantive examination  
GR01  Patent grant  
GR01  Patent grant 