CN108416347A - Well-marked target detection algorithm based on boundary priori and iteration optimization - Google Patents

Abstract

The invention discloses a salient target detection algorithm based on boundary prior and iterative optimization. Step 1, extracting feature image information, and expressing the image information in the form of a feature matrix; Step 2, establishing a region based on boundary prior Background likelihood estimation model, through which the position and contour of salient objects can be accurately detected; Step 3, generate a saliency map enhancement model based on iterative optimization, that is, iteratively perform two foreground/background seed selection and saliency global optimization deal with. Compared with the prior art, the salient object detection algorithm based on boundary prior and iterative optimization of the present invention combines various salient features and clues, and can greatly improve the quality of salient maps with any accuracy.

Description

Salient Object Detection Algorithm Based on Boundary Prior and Iterative Optimization

technical field

The present invention relates to the field of artificial intelligence and computer vision, more specifically, relates to a kind of image salient object detection algorithm.

Background technique

Salient target detection is one of the important topics in the field of computer vision. Its main task is to simulate the human visual attention mechanism and quickly segment the most attractive objects or regions from the image. At present, salient object detection has been used as an important image information preprocessing technology in many fields including image retrieval, object tracking, and object recognition. Visual saliency analysis can effectively guide the redundancy suppression of images, which is of great significance to image processing in the era of big data. However, due to the variety of objects in the image and the complex and diverse scenes, it is still a very challenging task to design a saliency analysis algorithm that can be applied to various scenes.

Salient object detection can quickly and accurately extract salient object regions. One of the ultimate goals of saliency detection is to reduce the amount of subsequent processing data to meet the challenges of massive image data. If the computational time complexity of the saliency detection algorithm itself is high, it will increase the burden of subsequent processing. And in many cases, although the image is very simple, the salient object detection algorithm cannot highlight the target and suppress the background well, so it cannot meet the accuracy requirements.

Contents of the invention

The purpose of the present invention is to propose a salient target detection algorithm based on boundary prior and iterative optimization in combination with scene depth information, by establishing a regional background likelihood estimation model based on boundary prior and a salient map enhancement model based on iterative optimization. Two stages are used to achieve salient object detection.

A kind of salient target detection algorithm based on boundary prior and iterative optimization of the present invention, this method comprises the following steps:

Step 1, extract feature image information, and represent the image information as a form of feature matrix; this step specifically includes the following processing:

First, perform image segmentation and region simplification: the input image is segmented using a simple linear iterative clustering algorithm, and each region is called a "superpixel", and the feature matrix formed by assigning the number of the superpixel to which each pixel belongs is obtained. ;

Secondly, according to the feature matrix, an undirected graph model G=(V, E) is established, where V represents the node set of the graph model, and E represents the undirected edge set.

Step 2. Establish a regional background likelihood estimation model based on the boundary prior, through which the position and contour of the salient target can be accurately detected. This step specifically includes the following processing:

First, establish a regional background likelihood estimation model based on the boundary prior, and the specific processing includes:

Find the homogeneity region H(ri ₎ of the superpixel r _i to be studied, and H(ri ₎ represents the superpixel set that is homogeneous with the superpixel r _i ;

Extract the boundary superpixel set B of the image;

Calculate the proportion of the boundary area that coincides with the homogeneous area of _ri in the boundary area, that is, the background likelihood of the superpixel _ri is defined as:

In the above formula, Indicates the background likelihood of the superpixel r _i , |·|indicates the total number of pixels in the superpixel or superpixel set;

Secondly, the homogeneity probability p _ij is estimated, and the estimation formula is as follows:

p _ij =M ^Cs (r _i , r _j )×M ^Con (r _i ,r _j )×M ^Sp (r _i )

Among them, i is the superpixel index to be estimated, j is the boundary superpixel index, M ^Cs (r _i , r _j ) is the color similarity of the superpixel pair (r _i , r _j ), M ^Con (r _i , r _j ) defines the connection smoothness between pairs of superpixels for the negative exponent of the geodesic distance, and M ^Sp ( _ri ) is a new central prior enhancement model. To facilitate subsequent calculations, use p _ij to construct a matrix where N _B is the total number of image boundary superpixels;

Furthermore, the estimation of the background image and the generation of the initial saliency map are realized, that is, the background image is generated according to the above-mentioned regional background likelihood estimation model, and converted into an elementary saliency map vector, as shown in the following formula:

in, Its elements are the normalized area sizes of all boundary superpixels, Indicates the area of the jth boundary superpixel.

vector Use a background likelihood probability map with the same resolution as the original image, where the part with a higher gray value represents the background area, and the part with a lower gray value represents the salient target area; the background image is reversed using the concept of Shannon's self-information is the initial saliency map; the self-information calculation formula is as follows:

A superpixel representing a lower background likelihood usually also contains more saliency information; use Indicates the initial salience degree of each superpixel i; i is the superpixel index to be estimated, j is the boundary superpixel index, and N is the number of superpixels.

Step 3. Generate a saliency map enhancement model based on iterative optimization, that is, iteratively perform two processes of foreground/background seed selection and saliency value global optimization, specifically including: in each iteration, first use a Bayesian theory-based The seed selection method extracts a small number of easily identifiable salient/background regions to form a seed set and And assign corresponding class labels to guide the subsequent optimization process; then use a least squares optimization model to fuse the three clues of class labels, prior estimates and smoothing priors, so that the output results are more accurate than the previous iteration input. sex and integrity. The model consists of an objective function and several constraints, and its expression is as follows:

In the t-th iteration, the saliency value of the superpixel r _i is denoted as The superscript (·) ^(t) in the above formula indicates that the variable is the variable in the t-th iteration; the objective function is the weighted sum of three least squares items, namely the prior item, the classification item and the smoothing item, and _δi are the adaptive weights in the t-th iteration, which are used to balance the above three items; in the constraints, is the label value for guiding classification. For the foreground seed, its value is 1, for the background seed, its value is 0, and the remaining superpixels can take arbitrary values.

Compared with the prior art, the salient object detection algorithm based on boundary prior and iterative optimization of the present invention combines a variety of salient features and clues, which can greatly improve the quality of salient maps with any accuracy; the optimization model of the present invention also It has strong versatility and error correction ability.

Description of drawings

Figure 1 is an example of background image and initial saliency image. (a) original image, (b) background image, (c) initial saliency image, (d) processing result of a first published algorithm MAP, (e) truth image;

Fig. 2 is a schematic diagram of the relationship between variables in a salient map enhancement model based on iterative optimization;

Fig. 3 is a schematic diagram of the background optimization process of a saliency map enhancement model based on iterative optimization; (a) the initial saliency map, (b) to (d) the saliency map after 1, 3, and 5 iterations of optimization and the corresponding average absolute Value error (MAE), (e) Truth map.

Fig. 4 is a schematic diagram of the saliency map optimization process of a saliency map enhancement model based on iterative optimization; (a) the original image and the true value map; (b) the randomly generated saliency map (the first and third lines) and the optimization using the patent model The results of (the second and fourth lines); (c) the saliency map estimated by using the Gaussian center prior model (the first and third lines) and the optimization results (the second and fourth lines); (d) the saliency map generated by the CA algorithm The saliency map (the first and third lines) and the optimization result (the second and the fourth line); (e) the saliency map (the first and the third line) and the optimization result (the second and the fourth line) generated by the FT algorithm; (f ) The saliency map (lines 1 and 3) and optimization results (lines 2 and 4) generated by the SVO algorithm;

FIG. 5 is a schematic diagram of the overall flow of the salient object detection algorithm based on boundary prior and iterative optimization of the present invention.

Detailed ways

Embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings.

As shown in Figure 5, it is an overall flowchart of a salient target detection algorithm based on boundary prior and iterative optimization of the present invention. The process specifically includes the following steps:

Step 1. In order to accurately identify the salient target position and contour in the image, extract some feature image information that is helpful for saliency analysis, and express the image information in the form of feature matrix. This step specifically includes the following processing:

First, perform image segmentation and region simplification: the Simple Linear Iterative Clustering (SLIC) algorithm is used to segment the input image into regions, and each region is called a "super-pixel". The algorithm flow is as follows:

Input: the number of superpixels K, the shape regularity coefficient m

1-1. Initialize each cluster center and sample pixels with a step size S;

1-2. Adjust the clustering center to the lowest point of the gradient (the direction of the neighborhood pixel with the largest difference between the pixel point and the 8 pixel points in the field) within a small local range (3×3 pixel block) (pixel The neighborhood pixel with the largest difference between the point and the 8 pixels in the domain);

1-3. Within the 2S×2S range of each cluster center, extract the LAB space color features [L _i , a _i , b _i ] and position features (xi _, y _i ) for pixel i, and calculate the pixel pair ( The distance of i, j) in the above feature space:

1-4. Perform K-means clustering on the pixels according to the distance d _ij to obtain a new cluster center;

1-5. Calculate the _L1 norm distance E between the new cluster center and the old cluster center;

1-6, stop iteration if E is less than a set threshold, otherwise repeat steps 1-3, 1-4;

Output: feature matrix, which is a matrix formed by assigning the number of the superpixel to which each pixel belongs.

Secondly, establish a graphical model: the constructed feature matrix independently describes the basic features of each region of the image. In order to further describe the relationship between superpixels, the present invention also establishes an undirected graphical model G=(V, E), where V represents the node set of the graph model, and E represents the undirected edge set. Consider each superpixel as a node in the undirected graph (represented as r _i for convenience), and the node pairs ( _ri , r _j ) satisfying the following conditions are connected:

(1) r _i and r _j are adjacent;

(2) Although r _i and r _j are not adjacent, both are adjacent to node r _k ;

(3) Both r _i and r _j are at the image boundary (including image boundary pixels).

It can be known from the above definition of node connection relationship that the undirected graph used in the present invention is a sparse graph. The matrix W=[w _ij ] _N×N is used to represent the similarity relationship between any superpixel pair (r _i , r _j ), then most of the elements in W are 0. In this patent, the similarity matrix is defined as follows:

in, Indicates the difference in average color between the i-th and j-th superpixels; Neig(r _i , r _j ) is a binarization function used to determine whether a node pair (r _i , r _j ) is connected, when r _i Neig( _ri ,r _j )=1 when connected to r _j , otherwise Neig( _ri ,r _j )=0; λ is a constant used to balance the weight of node connections.

Step 2. Establish a regional background likelihood estimation model based on the boundary prior, through which the position and contour of the salient target can be accurately detected:

First, establish a regional background likelihood estimation model based on the boundary prior. The specific establishment process is mainly divided into the following three parts:

2-1. Find the homogeneity region H(ri ₎ of the superpixel r _i to be studied, and H(ri ₎ represents the set of superpixels that are homogeneous to the superpixel r _i ;

2-2. Extracting the boundary superpixel set B of the image;

2-3. Calculate the proportion of the part of the boundary area that coincides with the homogeneous area of r _i in the boundary area, that is, the background likelihood of the superpixel r _i is defined as:

In the above formula, Indicates the background likelihood of the superpixel r _i , |·|indicates the total number of pixels in the superpixel or superpixel set.

Secondly, estimate the homogeneity probability p _ij : the homogeneity probability p _ij measures the possibility that a superpixel r _i and a certain border superpixel r _j belong to a homogeneous region, and is a key parameter in background detection. Considering the three factors of color similarity, connection smoothness and spatial proximity, the estimation formula of homogeneity probability p _ij is as follows:

p _ij =M ^Cs (r _i , r _j )×M ^Con (r _i ,r _j )×M ^Sp (r _i ) (4)

Among them, i is the superpixel index to be estimated, j is the boundary superpixel index, M ^Cs (r _i , r _j ) is the color similarity of the superpixel pair (r _i , r _j ), M ^Con (r _i , r _j ) defines the connection smoothness between pairs of superpixels (nodes) for the negative exponent of geodesic distance, and M ^Sp ( _ri ) is a new central prior enhancement model. To facilitate subsequent calculations, use p _ij to construct a matrix where N _B is the total number of image boundary superpixels. Inspired by the boundary prior, the present invention innovatively establishes a region-level background likelihood estimation model, thereby indirectly obtaining accurate prediction results of the saliency of all regions of the image.

Furthermore, the estimation of the background image and the generation of the initial saliency map are realized, that is, the background image is generated according to the above-mentioned regional background likelihood estimation model, and converted into an elementary saliency map: the homogeneity probability matrix P gives any superpixel r _i and a boundary superpixel With the probability of homogeneity features, the background likelihood of all superpixels is written as a vector in turn in, Denotes the background likelihood of superpixel r _i (Equation (3)) is the background likelihood of N superpixels Vector form (formula (5)), according to the definition formula of the background likelihood of superpixel r _i , the value of the vector is as follows:

in, Its elements are the normalized area sizes of all boundary superpixels, Indicates the area of the jth boundary superpixel.

vector It can be presented with a background likelihood probability map with the same resolution as the original image, as shown in Figure 1. The part with higher gray value in the figure represents the background region, and the part with lower gray value represents the salient target region. The meanings of the background map and the saliency map are just opposite, and this patent uses the concept of Shannon's self-information to invert the background map into the initial saliency map. Self-information is a good way to calculate saliency, which means that a superpixel with lower background likelihood usually also contains more saliency information. use Indicates the initial saliency of each superpixel, and its value can be calculated by the following expression,

It can be seen from Figure 1 that the method of the present invention can effectively describe the position and outline of the salient target. Although this patent only uses it as an initial estimate, its effect is already comparable to the current advanced technology.

Step 3. Generate a saliency map enhancement model based on iterative optimization. The optimization framework of this patent iteratively performs two steps: foreground/background seed selection and saliency value global optimization:

In each iteration, a seed selection method based on Bayesian theory is first used to extract a small number of easily identifiable salient/background regions to form a seed set and And assign corresponding class labels to guide the subsequent optimization process; then use a least squares optimization model to fuse the three clues of class labels, prior estimates and smoothing priors, so that the output result is more accurate than the input of the previous iteration. sex and integrity. The model consists of an objective function and several constraints, and its expression is as follows:

In the t-th iteration, the saliency value of the superpixel r _i is denoted as The superscript (·) ^(t) in the above formula indicates that the variable is the variable in the tth iteration. The objective function is a weighted sum of three least-squares terms, the prior term, the classification term, and the smoothing term, and _δi are the adaptive weights in the t-th iteration, which are used to balance the above three terms. Among the constraints, is the label value for guiding classification. For the foreground seed, its value is 1, for the background seed, its value is 0, and the remaining superpixels can take arbitrary values. The quality of the saliency map is qualitatively improved, and the accuracy and completeness of the initial estimation results are greatly improved. As shown in Figure 4, the saliency maps generated by random generation, Gaussian model, and three classical algorithms (CA, FT, SVO) are selected. After the model of the present invention is optimized, the output result can still obtain a high accuracy rate. Moreover, the present invention also has a strong error correction capability, and when the input result has extremely low precision or is even highly misleading, the method can also optimize a high-quality saliency map.

Claims (1)

1. A salient object detection algorithm based on boundary prior and iterative optimization is characterized by comprising the following steps:

step one, extracting characteristic image information, and representing the image information in a characteristic matrix form; the method specifically comprises the following steps:

first, image segmentation and region simplification are performed: performing region segmentation on an input image by adopting a simple linear iterative clustering algorithm, wherein each region is called a super pixel, and obtaining a characteristic matrix formed by distributing the serial number of the super pixel to each pixel;

secondly, establishing a undirected graph model G (V, E) according to the feature matrix, wherein V represents a node set of a graph model, and E represents an undirected edge set;

establishing a boundary prior-based region background likelihood estimation model, and accurately detecting the position and the contour of a significant target through the model, wherein the step specifically comprises the following steps:

firstly, establishing a boundary prior-based region background likelihood estimation model, and specifically processing the model comprises the following steps:

finding out superpixel r to be researched_iRegion of homogeneity H (r)_i)，H(r_i) Representation and superpixel r_iA homogenous set of superpixels;

extracting a boundary super-pixel set B of the image;

computing the sum r in the boundary region_iThe proportion of the overlapping portion of the homogeneous regions of (b) to the boundary region, i.e. the super-pixel r_iIs defined as:

in the above formula, the first and second carbon atoms are,representing a super pixel r_iThe background likelihood, |, represents the total number of pixels in a superpixel or superpixel set;

secondly, a homogeneity probability p is performed_ijThe estimation formula is as follows:

p_ij＝M^Cs(r_i,r_j)×M^Con(r_i,r_j)×M^Sp(r_i)

wherein i is the superpixel index to be estimated, j is the boundary superpixel index, M^Cs(r_i,r_j) Is a super pixel pair (r)_i,r_j) Color similarity of (1), M^Con(r_i,r_j) Defining smoothness of connection between superpixel pairs for negative index of geodesic distance, M^Sp(r_i) The method is a brand new central prior enhanced model; by p_ijConstructing a matrixWherein N is_BIs the total number of image border superpixels;

furthermore, the background map estimation and the initial saliency map generation are realized, that is, the background map is generated according to the above-mentioned region background likelihood estimation model and converted into an initial saliency map vector, as shown in the following formula:

wherein,its elements are the normalized area size of all boundary superpixels, represents the area of the jth boundary superpixel;

vector quantityPresenting the image by using a background likelihood probability map with the same resolution as the original image, wherein the part with the higher gray value represents a background area, and the part with the lower gray value represents a salient object area; the background image is inverted into an initial saliency image by utilizing the concept of Shannon self information; the self-information calculation formula is as follows:

a superpixel that represents a lower background likelihood will also typically contain more saliency information; by usingRepresenting the initial degree of saliency of each super-pixel i; i is a superpixel index to be estimated, j is a boundary superpixel index, and N is the number of superpixels;

step three, generating a saliency map enhancement model based on iterative optimization, namely iteratively executing two processes of foreground/background seed selection and saliency global optimization, specifically comprising: in each iteration, firstly, a seed selection method based on Bayesian theory is used to extract a few obvious/background regions which are easy to identify to form a seed setAndand endowing corresponding class labels to guide the subsequent optimization process; then, a least square optimization model is used for fusing three clues of class labels, prior estimation and smooth prior, so that the output result has higher accuracy and integrity than the last iteration input; the model consists of an objective function and a plurality of constraint conditions, and the expression of the model is as follows:

S_i ^(t+1)∈[0,1],S_i ^(t)∈[0,1]

in the t-th iteration, the super pixel r_iIs expressed asUpper middle label (·)^(t)Indicating that the variable is the variable in the t iteration; the objective function is a weighted sum of three least squares terms, i.e. a priori term, a classification term and a smoothing term,and delta_iIs the adaptive weight in the t iteration, is used for balancing the above three items; in the constraint condition, the number of the optical fiber,the label value of the guided classification is 1 for the foreground seed, 0 for the background seed, and the values of the rest super pixels can be arbitrarily selected.