CN107085725B - Method for clustering image areas through LLC based on self-adaptive codebook - Google Patents

Method for clustering image areas through LLC based on self-adaptive codebook Download PDF

Info

Publication number
CN107085725B
CN107085725B CN201710263353.4A CN201710263353A CN107085725B CN 107085725 B CN107085725 B CN 107085725B CN 201710263353 A CN201710263353 A CN 201710263353A CN 107085725 B CN107085725 B CN 107085725B
Authority
CN
China
Prior art keywords
super
llc
coding
pixel
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710263353.4A
Other languages
Chinese (zh)
Other versions
CN107085725A (en
Inventor
杨春蕾
普杰信
谢国森
刘中华
董永生
梁灵飞
司彦娜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Henan University of Science and Technology
Original Assignee
Henan University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Henan University of Science and Technology filed Critical Henan University of Science and Technology
Priority to CN201710263353.4A priority Critical patent/CN107085725B/en
Publication of CN107085725A publication Critical patent/CN107085725A/en
Application granted granted Critical
Publication of CN107085725B publication Critical patent/CN107085725B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/467—Encoded features or binary features, e.g. local binary patterns [LBP]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)

Abstract

The invention relates to a method for clustering image regions by LLC based on a self-adaptive codebook, which clusters adjacent regions by fused features after extracting features such as color, texture, Gabor, centroid and the like from segmented super pixels, improves the resolution between a plurality of adjacent regions in the form of a similarity structure chart, provides a very valuable clue for reducing and even eliminating noise near a foreground boundary in a generated saliency map and enables the foreground boundary to be clearer; the multiple features are used as a basis for calculating the saliency map, and under a complex scene, when the color features cannot effectively extract the saliency targets, the multi-feature information is used as a beneficial supplement, so that the detection effect can be effectively improved; the extended LLC coding scheme expands the method of respectively coding a plurality of feature descriptors in the original LLC and then fusing the feature descriptors into a method of fusing a plurality of feature descriptors and then coding once, simplifies the coding process and emphasizes the integrity of a plurality of features.

Description

Method for clustering image areas through LLC based on self-adaptive codebook
Technical Field
The invention relates to the technical field of pattern recognition technology, information fusion technology, information coding technology and digital image processing, in particular to a method for clustering image areas through LLC based on a self-adaptive codebook.
Background
The pattern recognition technology refers to a process of processing and analyzing various forms of (numerical, literal and logical relationship) information characterizing things or phenomena to describe, recognize, classify and explain the things or phenomena, and is an important component of information science and artificial intelligence. Pattern recognition in saliency detection refers to the recognition and classification of backgrounds and objects in images. A salient object is a person or thing in an image that stands out from the background, typically containing more interesting, more useful information. The main task of salient object detection is to detect and map the area where salient objects are located. Since the detection result can be directly used, the salient object detection is widely applied to the fields of object recognition, image segmentation, image retrieval and the like.
The commonly used salient object detection techniques mainly include salient region detection techniques based on local contrast, such as: based on local contrast and fuzzy growth technology, multi-scale center-periphery histogram and color space distribution contrast technology, etc.; and salient region detection techniques based on global contrast. The key in the salient object detection technology is to determine the salient value of each detection unit through the local or global feature difference among the detection units such as pixels, super-pixels, region blocks and the like, so feature extraction is a basic step for calculating the feature difference. Since the prominent color is the most fundamental feature that attracts human visual attention, one usually chooses the color to compute the feature difference. Although the performance of a plurality of existing salient object detection models is close to the standard of a test set in the scene of a single salient object and a simple background, the models cannot achieve better performance in the background of multiple objects and complex background, especially in the background of object fusion. When the image scene is complex, the color features may not be enough as a classification basis of the object and the background. This is because the complexity of the scene is typically manifested by the following characteristics: 1. the scene contains a plurality of objects with complex structures, and the objects may partially overlap with each other; 2. the target area is in an irregular shape; 3. the targets are distributed around the image; 4. the object has a similar hue to the background, or both have a cluttered hue. Among the above characteristics, the last characteristic is that it is difficult to extract the object from the background by using the color feature difference, and the texture feature difference can be used as an important basis for detecting the salient object. In addition, the objects located in the central area of the image are often noticed first, and the background is often distributed in the boundary area around the image, so that the advantage of the spatial relationship characteristic between the areas is highlighted, and the characteristic can also provide a referenceable clue for significance detection. When the color difference is not enough to provide clues for significant object detection, how to apply the multiple features of the image and effectively fuse them is a key issue to be solved. On the other hand, because the foreground is difficult to detect from a disordered background by the vision of the machine when the image scene is complex, the phenomena of more noise near the foreground region and even fuzzy foreground boundaries exist in the saliency maps generated by various advanced algorithms, and the difficulty of further foreground or target identification is improved.
Linear Coding (LLC) based on Locality-constrained is an efficient and robust classification technique, which was originally used mainly for image classification. The use of the method promotes the accuracy of image classification to be improved greatly due to the emphasis on the characteristic locality in the sparse coding process. Meanwhile, the LLC scheme has the characteristic of rapidity, the principle is simple, and the time required by coding is greatly shortened.
Feature Vector models (Feature Vector models) are widely used in the field of image processing. A plurality of feature data can be fused into one vector to be represented in a mode of 'uniform weight' or 'difference weight', and the representation method is simple and easy to participate in operation. The invention only relates to a vector model of 'uniform weight' to fuse the centroid, color, texture and Gabor features of an image region. Simple Linear Iterative Clustering (SLIC) is an efficient image segmentation method, in which an image is segmented into n super-pixels (the value of n generally has the best effect around 200), and pixels or image blocks divided into the same super-pixel have color similarity and internal compactness. At present, most of image saliency detection methods with better performance are based on SLIC superpixel segmentation, so that not only can the target of rapid detection be achieved, but also the obtained saliency map is smoother. At present, many efficient salient object detection algorithms use SLIC superpixels as basic detection units for feature extraction and salient value calculation.
Disclosure of Invention
The invention aims to provide a method for clustering image regions by LLC based on an adaptive codebook, which clusters adjacent regions by fused features after extracting features such as color, texture, centroid and the like from segmented super pixels, improves the resolution between a plurality of adjacent regions in a form of a similarity structure chart and provides a very valuable clue for reducing and even eliminating noise near a foreground boundary in a generated saliency map.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a method of clustering image regions by an adaptive codebook based LLC, comprising the steps of:
the method comprises the following steps: image over-segmentation and super-pixel region feature extraction: dividing an original image into n super pixels according to an SLIC method, and extracting the centroid, color, texture and Gabor characteristics of each super pixel region;
step two: and expanding LLC to carry out sparse coding on each super-pixel image region: according to the super-pixel feature extraction result, in order to emphasize the integrity of a plurality of features, for each super-pixel, according to the locality principle, firstly fusing a multi-feature descriptor, then performing single objective function optimization, organizing an adaptive codebook corresponding to each super-pixel, and encoding by using the adaptive codebook, thereby obtaining sparse encoding codewords of all super-pixel image regions;
step three: and (3) code conversion: converting the extended LLC coded code words into a symmetric square matrix to represent clustering results of adjacent regions;
step four: constructing a similarity structure chart: and constructing an image super pixel area similarity structure chart according to the result of code conversion and the characteristics of the super pixel area.
The image over-segmentation and super-pixel region feature extraction method in the first step comprises the following steps:
1) dividing an original image into n super pixels by using a SLIC method;
2) extracting the centroid of each superpixel of the original image, and expressing the centroid by using a horizontal coordinate and a vertical coordinate;
3) extracting three color mean values of each superpixel of the original image in a Lab space;
4) extracting three color mean values of each superpixel of the original image in an RGB space;
5) extracting LBP texture characteristic values of super pixels of an original image, and expressing the LBP texture characteristic values by 59 bins;
6) and extracting 36-dimensional Gabor characteristics of each superpixel of the original image.
The method for performing sparse coding on each super-pixel image region by expanding the LLC in the second step comprises the following steps:
1) for each super pixel, the color, texture and Gabor characteristic value are fused into a characteristic vector to represent the super pixel, and the super pixel is represented by formula (1) and stored:
Figure GDA0002539723930000031
2) for each super pixel spiCombining the color, texture and Gabor eigenvalues of all other superpixels except the superpixel in the image into a 101 × (n-1) -dimensional eigen matrix as the superpixel to be used as a codebook for LLC of the superpixel, wherein the codebook is an adaptive codebook
Figure GDA0002539723930000032
Expressed by equation (2):
Figure GDA0002539723930000033
3) performing extended LLC coding according to formulas (3) - (6) to obtain a coded code word of each super pixel corresponding to the dedicated codebook, writing a coding result into a (n-1) -dimensional coding indication vector, and defining and representing by a formula (7):
Figure GDA0002539723930000034
Figure GDA0002539723930000035
wherein the content of the first and second substances,
disti＝ρdist_ceni+(1-ρ)dist_lli(4)
dist_cenirepresents spiCentroid distance vector for each superpixel corresponding to codebook element:
Figure GDA0002539723930000036
dist_llirepresents spiFeature distance vector to codebook element:
Figure GDA0002539723930000037
represents spiDistance vectors to each element in the codebook, ⊙ denotes bit-wise multiplication of the corresponding elements between matrices or vectors;
recording indication vector of LLC coding:
Figure GDA0002539723930000041
wherein codeijRepresenting the coding coefficients of the pair i of superpixels j.
The method for code conversion in the third step is as follows:
1) the coded vector of each super pixel is expanded from dimension (n-1) to dimension n according to the following equation (8):
Figure GDA0002539723930000042
2) integrating all the coding vectors subjected to super-pixel expansion into an n multiplied by n coding matrix;
3) and (3) carrying out symmetry processing on the coding matrix, converting the coding matrix into a symmetric square matrix A according to the following formula (9) to represent clustering results, and finishing clustering representation of all superpixels:
Figure GDA0002539723930000043
wherein r isij＝0 and rjiNot equal to 0 indicates that if a superpixel i can represent j, but j is not contained in the nearest neighbor of j, then i and j should also be clustered into one class.
The method for constructing the similarity structure chart in the fourth step comprises the following steps:
1) finding out the positions indicating clustering relations in the symmetric coding matrix for clustering representation one by one, and calculating the similarity measurement value of the superpixels corresponding to the two nodes with the clustering relations according to the following formula (10):
Figure GDA0002539723930000044
wherein the content of the first and second substances,
Figure GDA0002539723930000045
and
Figure GDA0002539723930000046
representing a superpixel spiAnd spjRespectively representing Lab, RGB, lbp and Gabor, and psi representing all super-pixel sets containing image boundary pixels;
2) initializing an n multiplied by n all-zero matrix to represent an affine matrix of the similarity structure chart;
3) and extracting a position indicating that the clustering relation is 1 in a symmetrical coding matrix represented by the clusters, calculating four similarity measurement values of the corresponding two super pixels about the four features, and assigning the value of the affine matrix corresponding to the position as the product of the four similarity measurement values as a weight of an edge of the similarity structure chart to complete the construction of the similarity structure chart.
The invention has the beneficial effects that:
(1) the multiple features are used as a basis for calculating the saliency map, and under a complex scene, when the color features cannot effectively extract the saliency targets, the multi-feature information is used as a beneficial supplement, so that the detection effect can be effectively improved;
(2) the expanded LLC coding scheme expands the method of respectively coding a plurality of feature descriptors in the original LLC and then fusing the feature descriptors into a target function into the method of fusing a plurality of feature descriptors and then coding once, thereby simplifying the coding process and emphasizing the integrity of a plurality of features;
(3) the LLC is used once for each super pixel, the coding process is relatively independent and is not easily interfered by other adjacent regions, each coding is carried out around one super pixel region by the design of the self-adaptive codebook, and a clustering center can be automatically generated according to the occurrence frequency of the super pixels during coding conversion;
(4) the clustering result is shown in the form of a similarity structure chart, the graph structure of graph-based Manifold Ranking (GMR for short) can be directly improved, a more obvious performance improvement effect is obtained, the foreground boundary in the generated significant graph is clearer, and the noise is less; in three standard test libraries including SED2, ECSSD and DUT _ OMRON with complex scene image characteristics, the provided GroudTruth is used for sequentially comparing and calculating the saliency maps, Table 1 records the comparison results of the saliency maps obtained by using the map structure based on the improved manifold sorting of the invention on evaluation criteria such as average Fmeasure value (higher is better) and MAE value (lower is better), and the best two results are marked by bold fonts; the data in the table show that: the detection effect is obviously improved by using the LLC based on the multi-feature and self-adaptive codebook. Fig. 5 shows a comparison between a saliency detection map obtained based on the manifold ordering of the map structure improved map constructed by the present invention and saliency maps generated by other classical algorithms, and it can be seen that after the manifold ordering of the map structure improved map constructed by the present invention is used, the foreground boundary in the saliency detection map is clearer, the background noise is less, and the improvement effect is more obvious.
TABLE 1 Multi-Algorithm Performance comparison (eLLsC Algorithm uses the results of the present invention to construct a graph structure of manifold ordering)
eLLsC GMR GS SPL GR MNP SF GB MSS FT SR CA
DUT_OMRON
FMeasureF 0.5592 0.5298 0.443 0.5107 0.489 0.3976 0.4403 0.397 0.3587 0.2833 0.2405 0.4301
MAE 0.1563 0.1869 0.2101 0.1880 0.2557 0.2115 0.1895 0.2581 0.1770 0.2478 0.1786 0.254
ECSSD
FMeasureF 0.6950 0.6909 0.5876 0.6709 0.5206 0.5086 0.4694 0.5054 0.4491 0.3544 0.3299 0.3905
MAE 0.1883 0.1861 0.2327 0.2040 0.2829 0.2557 0.2228 0.2820 0.2446 0.2910 0.2674 0.3103
SED2
FMeasureF 0.7613 0.7286 0.6701 0.6605 0.7293 0.5451 0.706 0.4969 0.69 0.6307 0.4918 0.5456
MAE 0.1449 0.16301 0.1667 0.1628 0.1895 0.2225 0.1797 0.2418 0.1918 0.2057 0.2203 0.2292
Drawings
Fig. 1 is a general flow chart of a method for clustering image regions by an adaptive codebook-based LLC, to which the present invention relates;
FIG. 2 is a flow chart of an extended LLC to which the invention relates;
FIG. 3 is a transcoding flow diagram to which the present invention relates;
FIG. 4 is a flow diagram of generating a similarity structure graph in accordance with the present invention;
fig. 5 is a comparison graph of a saliency detection graph obtained based on the graph structure improvement graph manifold ordering constructed by the present invention and saliency maps generated by other classical algorithms.
Detailed Description
The invention is further illustrated with reference to specific embodiments below.
The invention relates to a method for clustering image areas through LLC based on an adaptive codebook, which comprises the following steps: dividing an original image area, extracting features, expanding LLC to perform sparse coding and coding conversion on each super-pixel image area, constructing a similarity structure chart and the like.
The super-pixel region segmentation method related by the invention adopts the current pixel clustering technology with better performance, namely the SLIC method, the clustered super-pixels are compact in interior, and the edges of the salient targets can be effectively stored, so that the finally generated salient image is ensured to smoothly and clearly display the target contour.
The super pixel feature extraction selects Lab color, RGB color, LBP texture and Gabor wavelet of the super pixel region of the image to construct a feature vector, and uses the centroid of the super pixel region to participate in expanding LLC coding criterion; the LBP texture and the Gabor wavelet can effectively distinguish the difference between the super pixels under the condition that a complex background or the background is similar to the target tone, and the characteristic locality of a neighboring region is highlighted by the centroid characteristic from the aspect of spatial relation.
The extended LLC scheme related by the invention is derived from the robust LLC which is commonly used for image classification, and the extended LLC scheme is mainly characterized in that: the method for respectively coding according to the multiple feature descriptors and then fusing the multiple feature descriptors into the objective function in the original LLC scheme is improved into the method for firstly fusing the multiple feature descriptors and then carrying out single objective function optimization, and the scheme of the self-adaptive codebook is used for coding.
The code conversion according to the present invention converts an (n-1) × n code matrix into an n × n square matrix, and performs a symmetry process on the matrix to show a clustering result of an image area.
The construction of the similarity structure chart provides an improved chart adjacency structure for an efficient significance detection method, namely manifold sequencing, so that the generated significance chart can ensure a clearer foreground boundary and reduce noise influence on the basis of smooth neighborhood significance.
To illustrate the method for clustering image regions by LLC based on adaptive codebook according to the present invention, the following is described with reference to the following embodiments and accompanying drawings:
fig. 1 is a general flow chart of the method of clustering image regions by LLC based on adaptive codebook according to the present invention. The method realizes the extraction of regional characteristics, LLC based on self-adaptive codebook, code conversion and clustering representation, the construction of similarity structure chart and the like through 6 basic steps, and comprises the following steps:
firstly, dividing an original image into n (the value of n is about 200) superpixels by using a SLIC algorithm;
extracting the characteristics of Lab color, RGB color, LBP texture, Gabor wavelet and the like of each super pixel region, and expressing and storing the characteristics by a formula (1);
(III) for each superpixel spiCreating its adaptive codebook
Figure GDA0002539723930000061
Expressed by formula (2);
(IV) performing extended LLC coding according to the formulas (3) to (6), and defining and representing by the formula (7);
(V) after LLC coding of all superpixels based on the adaptive codebook is completed, converting the coding matrix into a symmetrical square matrix A according to a formula (8) and a formula (9) to represent a clustering result;
calculating similarity measurement values among the super pixels with the clustering relation according to a formula (10), and creating an affine matrix taking the measurement values as weight values; four types of features are fused by integrating four affine matrices using equation (11), thereby creating a similarity structure graph.
Because the manifold sequencing result based on the graph is very sensitive to the adjacent structure of the adjacent graph, the graph structure created by clustering the image areas through the LLC based on the adaptive codebook can be used as the input of the manifold sequencing to generate the gray level saliency map with more accurate foreground boundary.
Super pixel region feature vector:
Figure GDA0002539723930000071
superpixel spiAdaptive codebook of (2):
Figure GDA0002539723930000072
LLC rule based on adaptive codebook:
Figure GDA0002539723930000073
Figure GDA0002539723930000074
wherein the content of the first and second substances,
disti＝ρdist_ceni+(1-ρ)dist_lli(4)
dist_cenirepresents spiCentroid distance vector for each superpixel corresponding to codebook element:
Figure GDA0002539723930000075
dist_llirepresents spiFeature distance vector to codebook element:
Figure GDA0002539723930000076
represents spiDistance vectors to each element in the codebook, ⊙ denotes the bitwise multiplication of the corresponding elements between matrices (or vectors).
Recording indication vector of LLC coding:
Figure GDA0002539723930000081
wherein codeijRepresenting the coding coefficients of the pair i of superpixels j.
Extension coding indication vector:
Figure GDA0002539723930000082
and (3) symmetry treatment:
Figure GDA0002539723930000083
wherein r isij＝0 and rjiNot equal to 0 indicates that if a superpixel i can represent j, but j is not contained in the nearest neighbor of j, then i and j should also be clustered into one class.
Similarity measure values for superpixels i and j:
Figure GDA0002539723930000084
wherein the content of the first and second substances,
Figure GDA0002539723930000085
and
Figure GDA0002539723930000086
representing a superpixel spiAnd spjDenotes Lab, RGB, lbp and Gabor, respectively, and Ψ denotes all superpixel sets containing image boundary pixels. Next, four types of features can be fused by integrating four affine matrices, thereby creating a graph structure:
W＝WLab⊙WRGB⊙Wlbp⊙WGabor(11)
an indication indicates a bit-wise multiplication of corresponding elements between matrices (or vectors).

Claims (5)

1. A method of clustering image regions by an LLC based on an adaptive codebook, comprising the steps of:
the method comprises the following steps: image over-segmentation and super-pixel region feature extraction: dividing an original image into n super pixels according to an SLIC method, and extracting the centroid, color, texture and Gabor characteristics of each super pixel region;
step two: and expanding LLC to carry out sparse coding on each super-pixel image region: according to the super-pixel feature extraction result, in order to emphasize the integrity of a plurality of features, for each super-pixel, according to the locality principle, firstly fusing a multi-feature descriptor, then performing single objective function optimization, organizing an adaptive codebook corresponding to each super-pixel, and encoding by using the adaptive codebook, thereby obtaining sparse encoding codewords of all super-pixel image regions;
step three: and (3) code conversion: converting the extended LLC coded code words into a symmetric square matrix to represent clustering results of adjacent regions;
step four: constructing a similarity structure chart: and constructing an image super pixel area similarity structure chart according to the result of code conversion and the characteristics of the super pixel area.
2. Method of clustering image regions by an adaptive codebook based LLC as claimed in claim 1, characterized in that: the image over-segmentation and super-pixel region feature extraction method in the first step comprises the following steps:
1) dividing an original image into n super pixels by using a SLIC method;
2) extracting the centroid of each superpixel of the original image, and expressing the centroid by using a horizontal coordinate and a vertical coordinate;
3) extracting three color mean values of each superpixel of the original image in a Lab space;
4) extracting three color mean values of each superpixel of the original image in an RGB space;
5) extracting LBP texture characteristic values of super pixels of an original image, and expressing the LBP texture characteristic values by 59 bins;
6) and extracting 36-dimensional Gabor characteristics of each superpixel of the original image.
3. Method of clustering image regions by an adaptive codebook based LLC as claimed in claim 1, characterized in that: the method for performing sparse coding on each super-pixel image region by expanding the LLC in the second step comprises the following steps:
1) for each super pixel, the color, texture and Gabor characteristic value are fused into a characteristic vector to represent the super pixel, and the super pixel is represented by formula (1) and stored:
Figure FDA0002539723920000011
2) for each super pixel spiCombining the color, texture and Gabor eigenvalues of all other superpixels except the superpixel in the image into a 101 × (n-1) -dimensional eigen matrix as the superpixel to be used as a codebook for LLC of the superpixel, wherein the codebook is an adaptive codebook
Figure FDA0002539723920000012
Expressed by equation (2):
Figure FDA0002539723920000013
3) performing extended LLC coding according to formulas (3) - (6) to obtain a coded code word of each super pixel corresponding to the dedicated codebook, writing a coding result into a (n-1) -dimensional coding indication vector, and defining and representing by a formula (7):
Figure FDA0002539723920000014
Figure FDA0002539723920000015
wherein the content of the first and second substances,
disti＝ρdist_ceni+(1-ρ)dist_lli(4)
dist_cenirepresents spiCentroid distance vector for each superpixel corresponding to codebook element:
Figure FDA0002539723920000021
dist_llirepresents spiFeature distance vector to codebook element:
Figure FDA0002539723920000022
represents spiDistance vectors to each element in the codebook, ⊙ denotes bit-wise multiplication of the corresponding elements between matrices or vectors;
recording indication vector of LLC coding:
Figure FDA0002539723920000023
wherein codeijRepresenting the coding coefficients of the pair i of superpixels j.
4. Method of clustering image regions by an adaptive codebook based LLC as claimed in claim 3, characterized in that: the method for code conversion in the third step is as follows:
1) the coded vector of each super pixel is expanded from dimension (n-1) to dimension n according to the following equation (8):
Figure FDA0002539723920000024
2) integrating all the coding vectors subjected to super-pixel expansion into an n multiplied by n coding matrix;
3) and (3) carrying out symmetry processing on the coding matrix, converting the coding matrix into a symmetric square matrix A according to the following formula (9) to represent clustering results, and finishing clustering representation of all superpixels:
Figure FDA0002539723920000025
wherein r isij＝0 and rjiNot equal to 0 indicates that if a superpixel i can represent j, but j is not contained in the nearest neighbor of j, then i and j should also be clustered into one class.
5. Method of clustering image regions by an adaptive codebook based LLC as claimed in claim 4, characterized in that: the method for constructing the similarity structure chart in the fourth step comprises the following steps:
1) finding out the positions indicating clustering relations in the symmetric coding matrix for clustering representation one by one, and calculating the similarity measurement value of the superpixels corresponding to the two nodes with the clustering relations according to the following formula (10):
Figure FDA0002539723920000026
wherein the content of the first and second substances,
Figure FDA0002539723920000031
and
Figure FDA0002539723920000032
representing a superpixel spiAnd spjRespectively representing Lab, RGB, lbp and Gabor, and psi representing all super-pixel sets containing image boundary pixels;
2) initializing an n multiplied by n all-zero matrix to represent an affine matrix of the similarity structure chart;
3) and extracting a position indicating that the clustering relation is 1 in a symmetrical coding matrix represented by the clusters, calculating four similarity measurement values of the corresponding two super pixels about the four features, and assigning the value of the affine matrix corresponding to the position as the product of the four similarity measurement values as a weight of an edge of the similarity structure chart to complete the construction of the similarity structure chart.
CN201710263353.4A 2017-04-21 2017-04-21 Method for clustering image areas through LLC based on self-adaptive codebook Active CN107085725B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710263353.4A CN107085725B (en) 2017-04-21 2017-04-21 Method for clustering image areas through LLC based on self-adaptive codebook

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710263353.4A CN107085725B (en) 2017-04-21 2017-04-21 Method for clustering image areas through LLC based on self-adaptive codebook

Publications (2)

Publication Number Publication Date
CN107085725A CN107085725A (en) 2017-08-22
CN107085725B true CN107085725B (en) 2020-08-14

Family

ID=59612198

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710263353.4A Active CN107085725B (en) 2017-04-21 2017-04-21 Method for clustering image areas through LLC based on self-adaptive codebook

Country Status (1)

Country Link
CN (1) CN107085725B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107622280B (en) * 2017-09-14 2020-06-12 河南科技大学 Modularized processing mode image saliency detection method based on scene classification
CN109166133B (en) * 2018-07-14 2021-11-23 西北大学 Soft tissue organ image segmentation method based on key point detection and deep learning

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103310195A (en) * 2013-06-09 2013-09-18 西北工业大学 LLC-feature-based weak-supervision recognition method for vehicle high-resolution remote sensing images
CN104239522A (en) * 2014-09-17 2014-12-24 西安电子科技大学 Inter-image content constraint-based ultra-pixel coding image retrieval method
CN104463882A (en) * 2014-12-15 2015-03-25 西安电子科技大学 SAR image segmentation method based on shape completion area chart and feature coding
EP2919193A2 (en) * 2014-03-12 2015-09-16 Nokia Technologies OY Method and apparatus for image segmentation
CN105118049A (en) * 2015-07-22 2015-12-02 东南大学 Image segmentation method based on super pixel clustering
CN105488812A (en) * 2015-11-24 2016-04-13 江南大学 Motion-feature-fused space-time significance detection method
CN105574534A (en) * 2015-12-17 2016-05-11 西安电子科技大学 Significant object detection method based on sparse subspace clustering and low-order expression
CN106203430A (en) * 2016-07-07 2016-12-07 北京航空航天大学 A kind of significance object detecting method based on foreground focused degree and background priori

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6451538B2 (en) * 2015-07-22 2019-01-16 富士通株式会社 Arithmetic processing device and control method of arithmetic processing device

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103310195A (en) * 2013-06-09 2013-09-18 西北工业大学 LLC-feature-based weak-supervision recognition method for vehicle high-resolution remote sensing images
EP2919193A2 (en) * 2014-03-12 2015-09-16 Nokia Technologies OY Method and apparatus for image segmentation
CN104239522A (en) * 2014-09-17 2014-12-24 西安电子科技大学 Inter-image content constraint-based ultra-pixel coding image retrieval method
CN104463882A (en) * 2014-12-15 2015-03-25 西安电子科技大学 SAR image segmentation method based on shape completion area chart and feature coding
CN105118049A (en) * 2015-07-22 2015-12-02 东南大学 Image segmentation method based on super pixel clustering
CN105488812A (en) * 2015-11-24 2016-04-13 江南大学 Motion-feature-fused space-time significance detection method
CN105574534A (en) * 2015-12-17 2016-05-11 西安电子科技大学 Significant object detection method based on sparse subspace clustering and low-order expression
CN106203430A (en) * 2016-07-07 2016-12-07 北京航空航天大学 A kind of significance object detecting method based on foreground focused degree and background priori

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
Saliency Aware Locality-Preserving Coding for Image Classification;Quan Fang 等;《2012 IEEE International Conference on Multimedia and Expo》;20121231;第260-265页 *
Saliency Detection via Graph-Based Manifold Ranking;Chuan Yang 等;《2013 IEEE Conference on Computer Vision and Pattern Recognition》;20131003;第1-8页 *
Salient object detection in complex scenes via D-S evidence theory based region classification;Chunlei Yang 等;《Vis Comput》;20160706;第1415-1428页 *
一种结合先验知识和半监督学习的显著性检测方法;王祥雒 等;《现代计算机》;20170415;第18-21、41页 *
基于区域再分割的改进型SLIC超像素生成算法;颜雨 等;《激光杂志》;20161231;第37卷(第8期);第129-133页 *
基于显著性稀疏表示的图像超分辨率算法;白蔚 等;《中国科技论文》;20140131;第9卷(第1期);第103-107页 *

Also Published As

Publication number Publication date
CN107085725A (en) 2017-08-22

Similar Documents

Publication Publication Date Title
CN109522908B (en) Image significance detection method based on region label fusion
CN108537239B (en) Method for detecting image saliency target
CN109086777B (en) Saliency map refining method based on global pixel characteristics
CN110334762B (en) Feature matching method based on quad tree combined with ORB and SIFT
CN108629783B (en) Image segmentation method, system and medium based on image feature density peak search
CN110866896B (en) Image saliency target detection method based on k-means and level set super-pixel segmentation
Huang et al. Saliency and co-saliency detection by low-rank multiscale fusion
CN106157330B (en) Visual tracking method based on target joint appearance model
Aytekin et al. Visual saliency by extended quantum cuts
KR101567792B1 (en) System and method for describing image outlines
CN107622280B (en) Modularized processing mode image saliency detection method based on scene classification
WO2010043771A1 (en) Detecting and tracking objects in digital images
Alamdar et al. A new color feature extraction method based on QuadHistogram
CN107067037B (en) Method for positioning image foreground by using LL C criterion
CN110738672A (en) image segmentation method based on hierarchical high-order conditional random field
CN107610136B (en) Salient object detection method based on convex hull structure center query point sorting
CN107085725B (en) Method for clustering image areas through LLC based on self-adaptive codebook
CN111091129A (en) Image salient region extraction method based on multi-color characteristic manifold sorting
Li et al. Edge-based split-and-merge superpixel segmentation
Schulz et al. Object-class segmentation using deep convolutional neural networks
LeBourgeois et al. Fast integral meanshift: application to color segmentation of document images
Basar et al. Color image segmentation using k-means classification on rgb histogram
Hassan et al. Salient object detection based on CNN fusion of two types of saliency models
CN112884884A (en) Candidate region generation method and system
CN112990226A (en) Salient object detection method based on machine learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant