CN102402621A - Image retrieval method based on image classification - Google Patents
Image retrieval method based on image classification Download PDFInfo
- Publication number
- CN102402621A CN102402621A CN2011104434345A CN201110443434A CN102402621A CN 102402621 A CN102402621 A CN 102402621A CN 2011104434345 A CN2011104434345 A CN 2011104434345A CN 201110443434 A CN201110443434 A CN 201110443434A CN 102402621 A CN102402621 A CN 102402621A
- Authority
- CN
- China
- Prior art keywords
- image
- retrieval
- classifier
- feature
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000012549 training Methods 0.000 claims abstract description 26
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 16
- 239000013598 vector Substances 0.000 claims description 35
- 238000012706 support-vector machine Methods 0.000 claims description 27
- 238000013507 mapping Methods 0.000 claims description 11
- 238000011161 development Methods 0.000 claims description 3
- 238000003064 k means clustering Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 238000007639 printing Methods 0.000 claims description 2
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 12
- 238000007796 conventional method Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 12
- 238000005457 optimization Methods 0.000 description 12
- 238000000605 extraction Methods 0.000 description 7
- 230000008859 change Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- 235000012149 noodles Nutrition 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000003066 decision tree Methods 0.000 description 2
- 230000009977 dual effect Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 241000282320 Panthera leo Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012887 quadratic function Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to an image retrieval method based on image classification and aims to solve the problem of lower retrieval speed by the conventional method. The method comprises the following steps of: firstly, determining the class number of images in the image classification and a training image set; secondly, extracting the content characteristics of the training image set for classifier training to obtain a classifier; thirdly, inputting an image to be retrieved, extracting the content characteristics of the image to be retrieved as the input of the classifier to obtain a retrieval image set which corresponds to the class, and extracting the content characteristics of each image in the retrieval image set; and finally, according to the obtained content characteristics, acquiring similarity distances of the image to be retrieved and each image in the retrieval image set by using a similarity calculation algorithm, sorting the distances to obtain N images which are nearest to the image to be retrieved, and outputting the N images. The image retrieval method based on the image classification has the advantage that: by combining an image classification technology and the conventional content-based image retrieval method, an image retrieval speed is greatly improved.
Description
Technical Field
The invention belongs to the field of multimedia information retrieval, and particularly relates to an image retrieval method based on image classification, which relates to the fields of image processing, computer vision, machine learning, image retrieval and the like, and can be directly used for content-based image retrieval in a Web environment and a single computer.
Background
With the development of the internet, multimedia information, particularly image information, is being generated and spread at an explosive speed. In the face of the huge image bank, how to mine images meeting the needs of the user becomes a very difficult problem, so the image retrieval field is becoming a hotspot of research and engineering practice in recent years. At present, there are two main methods for image retrieval: text-based image retrieval and content-based image retrieval.
The current mainstream image search engine adopts a text-based image retrieval technology, which is also a text retrieval technology in the traditional sense, and the process is to build indexes for text information related to network images, such as image titles, link texts, content descriptions and the like, to label the images, and finally, to retrieve by adopting a keyword matching technology. However, due to semantic ambiguity and descriptive ambiguity problems, the accuracy of text-based image retrieval techniques is relatively limited and the returned results are often not in line with the expectations of the user.
The content-based image retrieval is realized by introducing the technology in the field of computer vision to perform mathematical modeling and solving on an image pixel level object, so that the content characteristics of the image are used as the identification of the image. For the content feature extraction aspect of images, image retrieval currently has some transitions: the transformation from feature extraction based on the whole image to feature extraction based on the region (or object) makes the extracted features of the image more robust, keeps the features unchanged before and after the image is rotated and translated, and keeps relative invariance before and after the image illumination changes. The content-based image classification technology is a result of fusion of computer vision and pattern classification technology, and is a technology for automatically learning and classifying images by using a pattern classification method after extracting image content features. In the last decade, the classification technology is greatly improved, a support vector machine method based on a statistical probability theory is generated, and Adaboost methods established on a simple classifier at the bottom layer are generated, so that the classification accuracy of the methods is greatly improved compared with the prior methods such as a decision tree and a neural network. However, the original content-based image retrieval is difficult to be applied to a large-scale image library system due to large calculation amount, the application on the internet is less, and the image classification method only finds that the image belongs to a certain class and does not perform retrieval operation in the class.
Disclosure of Invention
The invention aims to provide an image retrieval method based on image classification, which integrates an image classification technology into image retrieval and solves the problem of slow retrieval speed in the current image retrieval based on content.
The invention provides an image retrieval method based on image classification, which comprises the following steps:
determining the number Cm of image categories in image classification, each detailed category (C1, C2, …, Cm) and training image sets (P1, P2, …, Pm) corresponding to each category;
b, extracting content characteristics of the training image set to train a classifier to obtain a classifier C;
inputting an image M to be retrieved, extracting content characteristics of the image M to be retrieved as the input of a classifier C, wherein the output result Co of the classifier C is the category of M;
d, obtaining a retrieval image set IPo corresponding to the category Co, and extracting the content characteristics of each image in the retrieval image set IPo;
and E, obtaining the similarity distance between the image M to be retrieved and each image in the retrieval image set IPo by using a similarity calculation algorithm according to the content characteristics obtained by C, D, sequencing the distances, and finally obtaining and outputting N images with the minimum distance to the image M to be retrieved.
Preferably, the image content features extracted in step B are SIFT features, and the specific steps include:
b11, extracting SIFT feature points of each image in the training image set to form a feature point library, wherein each SIFT feature point is represented by a 128-dimensional feature point descriptor vector;
b12, clustering the feature point library by adopting a K-means clustering method to obtain K classes;
b13, for a single image in the training image library, mapping all SIFT feature points of the single image to K classes respectively according to a minimum distance principle to obtain a K-dimensional feature vector V1, and dividing each value in the feature vector V1 by the total number of the feature points of the single image to obtain a frequency feature vector V2, wherein V2 is the SIFT feature of the single image.
Preferably, the classifier used in the step B is a support vector machine, and may be developed by using an open-source libsvm library, and the specific step of training the classifier includes:
b21, selecting RBF as a kernel function of the support vector machine;
b22, combining the feature vectors in the training image library with the category information of each image as the input of a support vector machine;
b23, determining the RBF kernel function parameters when the classification accuracy is highest by adopting a cross-printing method, and obtaining the support vector machine classifier C after determining the parameters.
Preferably, the specific step of extracting the content feature of the image M to be retrieved in step C includes:
c1, extracting SIFT feature points of the image M to be retrieved, wherein each SIFT feature point is represented by a 128-dimensional feature point descriptor vector;
and C2, mapping all SIFT feature points of the image M to be retrieved to K classes of B12 respectively according to a minimum distance principle to obtain a K-dimensional feature vector V1, and dividing each value in the feature vector V1 by the total number of the feature points of the image to obtain a frequency feature vector Vm, wherein the Vm is the SIFT feature of the M.
Preferably, the similarity calculation algorithm adopted in step E is a euclidean distance method, and the similarity measurement criterion is that smaller euclidean distances are more similar.
The invention has the beneficial effects that: the invention integrates the image classification technology on the basis of the traditional image retrieval method based on the content, thereby greatly improving the speed of image retrieval. In the content characteristic part of the image, SIFT characteristics are adopted, and the method has the characteristics of keeping invariance to scale scaling, brightness change and rotation of the image, so that the algorithm is more robust.
Drawings
In order to more clearly illustrate the embodiment of the present invention or the technical solutions in the prior art, the drawings used in the embodiment or the technical solutions in the prior art description will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
FIG. 1 is a flow diagram of the overall architecture of the present invention.
Fig. 2 is a process of constructing a gaussian difference scale space.
FIG. 3 is a schematic diagram of finding extreme points in scale space.
FIG. 4 is a diagram of feature vector generation by keypoint neighborhood gradient information.
Detailed Description
The present invention aims to provide an image retrieval method based on image classification, which integrates an image classification technology into image retrieval, solves the limitation of slow retrieval speed in the current content-based image retrieval, and specifically explains the implementation of the present invention with reference to the accompanying drawings in order to make the purpose, technical scheme and advantages of the present invention clearer.
Before describing the detailed flow of the present invention in detail, a brief introduction will be made to the basic knowledge required by the present invention:
a: SIFT content feature extraction of images
The SIFT content feature extraction of the image comprises two basic algorithms: extracting SIFT feature point descriptors and constructing image SIFT content features.
A1 extraction of SIFT feature point descriptor
The SIFT algorithm was proposed in d.g. lowe1999 and was perfected in 2004. The SIFT algorithm is an algorithm for extracting local features of an image and has the following characteristics: a) the SIFT features are local features of the image, and keep invariance to scale scaling, brightness change and rotation of the image, and also keep certain stability to radiation transformation, view angle change and noise. b) The method has the advantages of rich information quantity and good uniqueness, and is suitable for fast and accurate matching in a massive characteristic database. c) Even a few different images may produce a large number of different SIFT feature point descriptors. d) And the expandability can be conveniently fused with the feature vectors in other forms.
The SIFT feature point descriptor extraction algorithm comprises the following steps: 1) detecting extreme points on a scale space 2) accurately positioning the extreme points 3) calculating the direction parameters of each key point 4) generating descriptors of the key points. Each step is explained in detail below.
A11 Generation of a Scale space
The scale-space theory aims to model the multi-scale properties of image data.
The only linear kernel for realizing scale transformation is the Gaussian convolution kernel, and a two-dimensional image
The scale space of (a) is defined as:
wherein
Is a scale-variable gaussian function and is,
，
is a coordinate of the space, and is,
are scale coordinates.
In order to efficiently detect stable key points in scale space, the concept of the gaussian difference scale-space (DOG scale-space) is proposed. The difference of gaussians scale space is generated by convolution of different scales of difference of gaussians kernels and images.
As shown in fig. 2, the original scale-space model of the image is shown on the left side, and the gaussian difference space-scale model is shown on the right side.
A12 detection of spatial extreme points
In order to find the extreme point in the scale space, the sampling point needs to be compared with all its neighboring points to determine whether it is larger or smaller than all its neighboring points in the image domain and the scale domain. As shown in fig. 3, the middle detection point is compared with 26 points, which are 8 adjacent points of the same scale and 9 x 2 points of the upper and lower adjacent scales, to ensure that the extreme points are detected in both scale space and two-dimensional image space.
In order to accurately determine the position of the extreme point, the position and the scale of the key point are accurately determined by fitting a three-dimensional quadratic function, and meanwhile, the key point with low contrast and the unstable edge response point are removed, so that the matching stability is enhanced, and the anti-noise capability is improved.
A13 Key Point Direction Allocation
In order to enable the operator to have rotation invariance, a direction parameter is assigned to each key point by using the gradient direction distribution characteristic of the pixels in the neighborhood of the key point.
The two formulas are as above
The modulus of the gradient and the direction formula. WhereinThe scale used is the scale at which each keypoint is located. In actual calculation, sampling is carried out in a neighborhood window with a key point as the center, and the gradient direction of a neighborhood pixel is counted by using a histogram. The gradient histogram ranges from 0 to 360 degrees with one bin every 10 degrees for a total of 36 bins. The peak of the histogram represents the main direction of the neighborhood gradient at the keypoint, i.e. the direction that is taken as the keypoint. In the gradient direction histogram, when there is another peak corresponding to 80% of the energy of the main peak, this direction is considered as a secondary direction of the keypoint. A keypoint may be assigned multiple directions (one primary direction, more than one secondary direction), which may enhance the robustness of the match. So far, the detection of the key points of the image is finished, and a SIFT feature region can be determined.
A14 feature Point descriptor Generation
To ensure rotational invariance, the coordinate axes are first rotated to the direction of the key point. Next, a 8 x 8 window is taken centered on the keypoint. The central black point on the left part of fig. 4 is the position of the current keypoint, each square in the window represents a pixel in the scale space where the keypoint neighborhood is located, the length of the arrow represents the diyu module value, the direction of the arrow represents the gradient direction of the pixel, and the blue circle in the graph represents the range of gaussian weighting (pixels closer to the keypoint contribute more to the directional information). Then, a histogram of gradient directions in 8 directions is calculated on each 4 × 4 small block to obtain an accumulated value of each gradient direction, and a target point can be formed, as shown in the right part of the figure. In this figure, a key point consists of four target points 2 x 2, each having 8 pieces of direction vector information. The thought of neighborhood directivity information combination enhances the anti-noise capability of the algorithm, and simultaneously provides better fault tolerance for the feature matching containing the positioning error.
In the actual calculation process, in order to enhance the robustness of matching, Lowe proposes to use 16 seed points of 4 × 4 for each keypoint, so that 128 data can be generated for one keypoint, i.e., a 128-dimensional SIFT feature vector is finally formed. At this time, the SIFT feature vector has already removed the influence of set deformation factors such as scale change, rotation, etc., and then the length of the feature vector is normalized, so that the influence of illumination change can be further removed.
A2 image SIFT content feature structure
After the SIFT feature point descriptor subset of each image is obtained by the above method, because the SIFT feature points of each image are different and cannot be used in a general classification method, a method called bag of words is also needed to obtain a feature quantity with a fixed length to represent each image. The word package concept is as follows: regarding each feature point descriptor as a word, so that all feature point sets of the image set are a word bank, and dividing all words of the word bank into N classes by using a clustering method for the word bank; for a single image in the image set, the distribution of all the feature points of the single image in the N classes is counted to obtain an N-dimensional frequency distribution vector, and the vector is the SIFT content feature of the single image.
B: support vector machine classifier
An important topic in data mining is data classification. The data classification means that a classifier is formed by training according to a certain classification principle on the basis of the existing training data; the classifier is then used to determine the class of the unclassified data. Support vector machines are a classification boundary-based approach. The basic principle is (taking two-dimensional data as an example): if the training data is abstracted as points distributed on a two-dimensional plane, they are grouped into different regions on the plane according to their categories. The purpose of classification boundary-based classification algorithms is to find the boundaries between these classes through training. For multidimensional data (e.g., N-dimensional), they can be considered points in an N-dimensional space, while classification boundaries are hyper-surfaces in the N-dimensional space. The linear classifier uses a hyperplane type of boundary and the nonlinear classifier uses a hypersurface.
B1 basic principle of support vector machine classification
The support vector machine is based on linear partitioning. Not all data may be linearly partitioned. Points of two classes, such as in two-dimensional space, may require a curve as their boundary. Support vector machines take the measure of mapping points in a low-dimensional space into a high-dimensional space in order to make them linearly separable. In the high-dimensional space, the method is a linear division, and in the original data space, the method is a non-linear division. And judging the classification boundary by using a linear division method. When discussing specific algorithms for support vector machines, we will consider the optimization problem (finding the optimal solution for a certain objective) as how to define the mapping algorithm from low-dimensional to high-dimensional space is already included in its kernel function.
B2 optimization problem
If we represent a problem as a function
The optimization problem is to find the minimum or maximum of the function. In higher mathematics, if the function is continuously derivable, its extreme value can be found by taking the derivative, calculating the point where the derivative is zero. But in reality the functions are not necessarily continuously derivable. The optimization problem is a discussion of this situation.
The optimization problem can be divided into two categories: (1) an unconstrained optimization problem; (2) there is a constrained optimization problem.
The unconstrained optimization algorithm can be expressed as:
. The first approximate optimal solution can be obtained by a Newton method, a steepest gradient descent method and the like in a numerical calculation method through multiple cycles.
There is a constraint problem, generally expressed as:
b3 Linear divisible binary problem
The implications of the linearly separable binary problem are: the raw data may be divided by a straight line (if the data is only two-dimensional) or a hyperplane. There are three basic approaches to separating data into two classes using hyperplanes in a multidimensional space:
(1) square nearest point method: the bisector of the connecting line between the two closest points of the two types of points is used as a classification line (plane).
(2) Maximum interval method: and calculating a classification surface to maximize the interval of classification boundaries. The classification boundary is the translation of a value from the classification face to the points of the two classes, respectively, until the first data point is encountered. The distance between the classification boundaries of the two classes is the classification interval.
The classification plane is represented as:
. It is noted that,
is a multi-dimensional vector. The reciprocal of the classification interval is:
. The optimization problem is expressed as:
the constraint is as follows: require each data point
The distance to the classification surface is greater than or equal to 1. Wherein,is the classification of the data.
(3) Linear support vector classifier: classifying the noodles:
and, requiring:
based on this, find
(optimal solution, algorithm otherwise):
description of the drawings: the linear support vector machine is based on the maximum spacing method. The problem is a quadratic programming problem, the lagrangian function is used for combining the optimization problem and the constraint, and then the dual theory is used for obtaining the classification optimization problem. It should be noted that this problem is still a constrained optimization problem.
B4 Linear inseparable problem
(1) Linear soft interval classifier
The basic idea is as follows: the original requirement for the spacing cannot be met because the samples are linearly indistinguishable. Introducing relaxation variables
The constraint is weakened to:
. However, we still want this relaxation variable
Minimize (if0, the original linear hard interval classifier). Thus, penalty parameters are used in optimizing the objective function
To introduce into
The goal of minimization. Thus, the model of the classifier is: classifying the noodles:
and, requiring:
taking this as the original problem, the dual problem is:
(2) nonlinear hard interval classifier
The basic idea is as follows: a curve (surface) in a low-dimensional space can be mapped to a straight line or a plane in a high-dimensional space. The data is linearly separable in a high-dimensional space after being subjected to such mapping. Let the mapping be:
then, the linear support vector machine model in the high-dimensional space is:
classifying the noodles:and, requiring:
it should be noted that since the data is mapped to a high dimensional space，
Calculated quantity ratio ofMuch larger. At this time, a so-called "kernel function" is introduced:
as can be seen from the above formula, the kernel function has the effect that
While mapping to the high-dimensional space, the inner product of the two data in the high-dimensional space is calculated, so that the calculated amount is returned to
Of the order of magnitude of (d).
B5 Multi-Classification problem
(1) One class is to the rest class method:
a support vector machine is established that separates one class from the rest. If the training data has M types, M support vector machines need to be established. IdentificationWhen classifying, selectingThe largest classification:
or:calculate two maximums
As a confidence. If it is notThen selectThe largest class; otherwise, the classification is rejected.
(2) Paired classification
And (3) establishing classifiers between any two of the M classes, wherein M (M-1)/2 classifiers are shared. IdentificationWhen classifying, a voting method is adopted, and the class with the most votes isThe final classification of (1).
Having described the basic knowledge required by the present invention, the steps of the present invention will now be described in detail with reference to the flow chart of FIG. 1.
First, the classifier is a key component used in each search in the present invention. Since the classifier used in the present invention is a support vector machine, it is necessary to determine the number Cm of classes of images in image classification, each detailed class (C1, C2, …, Cm), and a training image set (P1, P2, …, Pm) corresponding to each class. The size of Cm is selected according to the application of the invention, and for the application of a larger search image set, such as image search on the Web, Cm should be larger, and the classification details the scale of the target search image. For each detailed category (C1, C2, …, Cm), different description methods are also adopted according to different application occasions, for example, in Web retrieval, a certain item Ci may correspond to some described keywords, and in some small local image retrieval systems, Ci may directly correspond to the file storage address of a certain type of image.
After determining the number of classes Cm of the images in the image classification, each detailed class (C1, C2, …, Cm) and the training image set (P1, P2, …, Pm) corresponding to each class, SIFT feature points of each image in the training image set are extracted by the method described above to form a feature point library, wherein each SIFT feature point is represented by a 128-dimensional feature point description sub-vector. And clustering the feature point library by adopting a K-means clustering method to obtain K classes. For a single image in a training image library, mapping all SIFT feature points of the single image to K classes respectively according to a minimum distance principle to obtain a K-dimensional feature vector V1, dividing each value in V1 by the total number of the feature points of the single image to obtain a frequency feature vector V2, wherein V2 is the SIFT content feature of the single image.
After the SIFT content features of the training image library are obtained, the support vector machine classifier is trained by using the content features. For support vector machines, development can be performed using an open-source libsvm library. Firstly, RBF is selected as a kernel function of a support vector machine, then, feature vectors in a training image library are combined with class information of each image to serve as input of the support vector machine, a cross-validation method is adopted to determine RBF kernel function parameters when the classification accuracy is highest, and a support vector machine classifier C is obtained after the parameters are determined.
After the support vector machine classifier C is obtained, the preparation work of the system is also completed. For an image M to be retrieved, firstly extracting the SIFT image content characteristics, and the steps are as follows: extracting the SIFT feature points of M, wherein each SIFT feature point is represented by a 128-dimensional feature point descriptor vector, mapping all SIFT feature points of M to K classes respectively according to a minimum distance principle to obtain a K-dimensional feature vector V1, and dividing each value in V1 by the total number of the feature points of the image to obtain a frequency feature vector Vm, wherein the Vm is the SIFT content feature of M. Support vector machine C accepts Vm as an input, resulting in a class Co of M. And obtaining IPo retrieval image sets corresponding to the categories Co, and extracting IPo SIFT content features of each image to form a content feature query set. And (3) calculating the Euclidean distance of each characteristic quantity in the Vm and IPo content query sets by using a Euclidean distance method, wherein the similarity measurement criterion is that the smaller the Euclidean distance is, the more similar the Euclidean distance is, sorting the distances to obtain and outputting N images with the minimum distance to M.
The content features and the classifier used in the above embodiments are only used for illustrating the present invention and are not limited to the present invention, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, such as the method of extracting content features can be extended to texture symbiotic matrix method, wavelet transform method, etc., and the classifier can select other methods such as neural network, decision tree, etc., as all equivalent technical solutions also fall into the scope of the present invention, and the patent protection scope of the present invention should be defined by the claims.
Claims (5)
1. An image retrieval method based on image classification is characterized by comprising the following steps:
determining the number Cm of image categories in image classification, each detailed category (C1, C2, …, Cm) and training image sets (P1, P2, …, Pm) corresponding to each category;
b, extracting content characteristics of the training image set to train a classifier to obtain a classifier C;
inputting an image M to be retrieved, extracting content characteristics of the image M to be retrieved as the input of a classifier C, wherein the output result Co of the classifier C is the category of M;
d, obtaining a retrieval image set IPo corresponding to the category Co, and extracting the content characteristics of each image in the retrieval image set IPo;
and E, obtaining the similarity distance between the image M to be retrieved and each image in the retrieval image set IPo by using a similarity calculation algorithm according to the content characteristics obtained by C, D, sequencing the distances, and finally obtaining and outputting N images with the minimum distance to the image M to be retrieved.
2. The image retrieval method according to claim 1, characterized in that: the content features extracted in the step B are SIFT features, and the method specifically comprises the following steps:
b11, extracting SIFT feature points of each image in the training image set to form a feature point library, wherein each SIFT feature point is represented by a 128-dimensional feature point descriptor vector;
b12, clustering the feature point library by adopting a K-means clustering method to obtain K classes;
b13, for a single image in the training image library, mapping all SIFT feature points of the single image to K classes respectively according to a minimum distance principle to obtain a K-dimensional feature vector V1, and dividing each value in the feature vector V1 by the total number of the feature points of the single image to obtain a frequency feature vector V2, wherein V2 is the SIFT feature of the single image.
3. The image retrieval method according to claim 1, characterized in that: the classifier adopted in the step B is a support vector machine, an open-source libsvm library can be adopted for development, and the specific steps of training the classifier comprise:
b21 selecting RBF as kernel function of support vector machine
B22, combining the feature vectors in the training image library with the category information of each image as the input of a support vector machine;
b23, determining the RBF kernel function parameters when the classification accuracy is highest by adopting a cross-printing method, and obtaining the support vector machine classifier C after determining the parameters.
4. The image retrieval method according to claim 1, characterized in that: the specific step of extracting the content features of the image M to be retrieved in the step C includes:
c1, extracting SIFT feature points of the image M to be retrieved, wherein each SIFT feature point is represented by a 128-dimensional feature point descriptor vector;
and C2, mapping all SIFT feature points of the image M to be retrieved to K classes of B12 respectively according to a minimum distance principle to obtain a K-dimensional feature vector V1, and dividing each value in the feature vector V1 by the total number of the feature points of the image to obtain a frequency feature vector Vm, wherein the Vm is the SIFT feature of the M.
5. The image retrieval method according to claim 1, characterized in that: and E, adopting a similarity calculation algorithm as a Euclidean distance method, wherein the similarity measurement criterion is that the smaller the Euclidean distance is, the more similar the Euclidean distance is.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011104434345A CN102402621A (en) | 2011-12-27 | 2011-12-27 | Image retrieval method based on image classification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011104434345A CN102402621A (en) | 2011-12-27 | 2011-12-27 | Image retrieval method based on image classification |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102402621A true CN102402621A (en) | 2012-04-04 |
Family
ID=45884821
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011104434345A Pending CN102402621A (en) | 2011-12-27 | 2011-12-27 | Image retrieval method based on image classification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102402621A (en) |
Cited By (35)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020111A (en) * | 2012-10-29 | 2013-04-03 | 苏州大学 | Image retrieval method based on vocabulary tree level semantic model |
| CN103177264A (en) * | 2013-03-14 | 2013-06-26 | 中国科学院自动化研究所 | Image classification method based on visual dictionary global topological representations |
| CN103177110A (en) * | 2013-03-28 | 2013-06-26 | 百度在线网络技术（北京）有限公司 | Whole set image searching method and whole set image searching equipment |
| CN103207910A (en) * | 2013-04-08 | 2013-07-17 | 河南大学 | Image retrieval method based on hierarchical features and genetic programming relevance feedback |
| CN103310235A (en) * | 2013-05-31 | 2013-09-18 | 中国科学院信息工程研究所 | Steganalysis method based on parameter identification and estimation |
| CN103377381A (en) * | 2012-04-26 | 2013-10-30 | 富士通株式会社 | Method and device for identifying content attribute of image |
| CN103425996A (en) * | 2013-08-01 | 2013-12-04 | 华南理工大学 | Parallel distribution type large-scale image recognition method |
| CN103473545A (en) * | 2013-08-01 | 2013-12-25 | 西安交通大学 | Text-image similarity-degree measurement method based on multiple features |
| CN103793466A (en) * | 2013-12-20 | 2014-05-14 | 深圳先进技术研究院 | Image retrieval method and image retrieval device |
| CN103902965A (en) * | 2012-12-29 | 2014-07-02 | 深圳先进技术研究院 | Spatial co-occurrence image representing method and application thereof in image classification and recognition |
| CN104392439A (en) * | 2014-11-13 | 2015-03-04 | 北京智谷睿拓技术服务有限公司 | Image similarity confirmation method and device |
| CN104504049A (en) * | 2014-12-20 | 2015-04-08 | 辽宁师范大学 | Retrieval method of color images based on quaternion Harmonic-Fourier moments |
| CN104504050A (en) * | 2014-12-20 | 2015-04-08 | 辽宁师范大学 | Retrieval method of color images based on quaternion invariants |
| CN105589929A (en) * | 2015-12-09 | 2016-05-18 | 东方网力科技股份有限公司 | Image retrieval method and device |
| CN105631037A (en) * | 2015-12-31 | 2016-06-01 | 北京恒冠网络数据处理有限公司 | Image retrieval method |
| CN105844283A (en) * | 2015-01-16 | 2016-08-10 | 阿里巴巴集团控股有限公司 | Method for identifying category of image, image search method and image search device |
| CN105975497A (en) * | 2016-04-27 | 2016-09-28 | 清华大学 | Automatic microblog topic recommendation method and device |
| CN106056159A (en) * | 2016-06-03 | 2016-10-26 | 西安电子科技大学 | Image fine classification method based on Fisher Vector |
| CN106203461A (en) * | 2015-05-07 | 2016-12-07 | 中国移动通信集团公司 | A kind of image processing method and device |
| CN106203537A (en) * | 2016-08-31 | 2016-12-07 | 成都铁安科技有限责任公司 | A kind of current collecting bow lifting condition detection method and device |
| CN106557526A (en) * | 2015-09-30 | 2017-04-05 | 富士通株式会社 | The apparatus and method for processing image |
| CN106651935A (en) * | 2016-11-29 | 2017-05-10 | 河南科技大学 | Multi-scale sampling-based texture image representation method |
| CN108805148A (en) * | 2017-04-28 | 2018-11-13 | 富士通株式会社 | Handle the method for image and the device for handling image |
| CN109033308A (en) * | 2018-07-16 | 2018-12-18 | 安徽江淮汽车集团股份有限公司 | A kind of image search method and device |
| CN109271533A (en) * | 2018-09-21 | 2019-01-25 | 深圳市九洲电器有限公司 | A kind of multimedia document retrieval method |
| CN109276243A (en) * | 2018-08-31 | 2019-01-29 | 易念科技（深圳）有限公司 | Brain electricity psychological test method and terminal device |
| CN109446356A (en) * | 2018-09-21 | 2019-03-08 | 深圳市九洲电器有限公司 | A kind of multimedia document retrieval method and device |
| CN109816025A (en) * | 2019-01-29 | 2019-05-28 | 宝鸡文理学院 | A kind of image search method based on image classification |
| CN110069654A (en) * | 2019-03-15 | 2019-07-30 | 平安城市建设科技(深圳)有限公司 | Source of houses searching method, device, equipment and computer readable storage medium |
| CN111339343A (en) * | 2020-02-12 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Image retrieval method, device, storage medium and equipment |
| CN111353011A (en) * | 2020-02-27 | 2020-06-30 | 北京市商汤科技开发有限公司 | Location data set, building method and device thereof, and data processing method and device |
| CN113868440A (en) * | 2020-06-30 | 2021-12-31 | 华为技术有限公司 | Method, device, equipment and medium for managing feature library |
| CN114611400A (en) * | 2022-03-18 | 2022-06-10 | 河北金锁安防工程股份有限公司 | Early warning information screening method and system |
| WO2022166604A1 (en) * | 2021-02-04 | 2022-08-11 | 腾讯科技（深圳）有限公司 | Image processing method and apparatus, computer device, storage medium, and program product |
| CN114926471A (en) * | 2022-05-24 | 2022-08-19 | 北京医准智能科技有限公司 | Image segmentation method and device, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101211341A (en) * | 2006-12-29 | 2008-07-02 | 上海芯盛电子科技有限公司 | Image intelligent mode recognition and searching method |
| CN101551809A (en) * | 2009-05-13 | 2009-10-07 | 西安电子科技大学 | Search method of SAR images classified based on Gauss hybrid model |
| US20100158356A1 (en) * | 2008-12-22 | 2010-06-24 | Yahoo! Inc. | System and method for improved classification |
| CN101859326A (en) * | 2010-06-09 | 2010-10-13 | 南京大学 | Image searching method |
| US20110052063A1 (en) * | 2009-08-25 | 2011-03-03 | Xerox Corporation | Consistent hierarchical labeling of image and image regions |
-
2011
- 2011-12-27 CN CN2011104434345A patent/CN102402621A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101211341A (en) * | 2006-12-29 | 2008-07-02 | 上海芯盛电子科技有限公司 | Image intelligent mode recognition and searching method |
| US20100158356A1 (en) * | 2008-12-22 | 2010-06-24 | Yahoo! Inc. | System and method for improved classification |
| CN101551809A (en) * | 2009-05-13 | 2009-10-07 | 西安电子科技大学 | Search method of SAR images classified based on Gauss hybrid model |
| US20110052063A1 (en) * | 2009-08-25 | 2011-03-03 | Xerox Corporation | Consistent hierarchical labeling of image and image regions |
| CN101859326A (en) * | 2010-06-09 | 2010-10-13 | 南京大学 | Image searching method |
Non-Patent Citations (2)
| Title |
|---|
| 张颖彬等: "基于SVM的相关反馈图像分类和检索方案", 《铁路计算机应用》 * |
| 汤杨等: "基于分级mean shift的图像分割算法", 《计算机研究与发展》 * |
Cited By (56)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103377381A (en) * | 2012-04-26 | 2013-10-30 | 富士通株式会社 | Method and device for identifying content attribute of image |
| CN103377381B (en) * | 2012-04-26 | 2016-09-28 | 富士通株式会社 | The method and apparatus identifying the contents attribute of image |
| CN103020111A (en) * | 2012-10-29 | 2013-04-03 | 苏州大学 | Image retrieval method based on vocabulary tree level semantic model |
| CN103902965A (en) * | 2012-12-29 | 2014-07-02 | 深圳先进技术研究院 | Spatial co-occurrence image representing method and application thereof in image classification and recognition |
| CN103902965B (en) * | 2012-12-29 | 2017-06-16 | 深圳先进技术研究院 | Spatial domain symbiosis image representing method and its application in image classification, identification |
| CN103177264A (en) * | 2013-03-14 | 2013-06-26 | 中国科学院自动化研究所 | Image classification method based on visual dictionary global topological representations |
| CN103177264B (en) * | 2013-03-14 | 2016-09-14 | 中国科学院自动化研究所 | The image classification method that view-based access control model dictionary Global Topological is expressed |
| CN103177110A (en) * | 2013-03-28 | 2013-06-26 | 百度在线网络技术（北京）有限公司 | Whole set image searching method and whole set image searching equipment |
| CN103177110B (en) * | 2013-03-28 | 2016-08-24 | 百度在线网络技术（北京）有限公司 | The method and apparatus searching for complete image |
| CN103207910A (en) * | 2013-04-08 | 2013-07-17 | 河南大学 | Image retrieval method based on hierarchical features and genetic programming relevance feedback |
| CN103207910B (en) * | 2013-04-08 | 2017-02-08 | 河南大学 | Image retrieval method based on hierarchical features and genetic programming relevance feedback |
| CN103310235A (en) * | 2013-05-31 | 2013-09-18 | 中国科学院信息工程研究所 | Steganalysis method based on parameter identification and estimation |
| CN103310235B (en) * | 2013-05-31 | 2016-04-27 | 中国科学院信息工程研究所 | A kind of steganalysis method based on parameter identification and estimation |
| CN103473545A (en) * | 2013-08-01 | 2013-12-25 | 西安交通大学 | Text-image similarity-degree measurement method based on multiple features |
| CN103425996B (en) * | 2013-08-01 | 2017-08-25 | 华南理工大学 | A kind of large-scale image recognition methods of parallel distributed |
| CN103473545B (en) * | 2013-08-01 | 2016-06-29 | 西安交通大学 | A kind of text image method for measuring similarity based on multiple features |
| CN103425996A (en) * | 2013-08-01 | 2013-12-04 | 华南理工大学 | Parallel distribution type large-scale image recognition method |
| CN103793466B (en) * | 2013-12-20 | 2018-06-05 | 深圳先进技术研究院 | A kind of image search method and device |
| CN103793466A (en) * | 2013-12-20 | 2014-05-14 | 深圳先进技术研究院 | Image retrieval method and image retrieval device |
| CN104392439A (en) * | 2014-11-13 | 2015-03-04 | 北京智谷睿拓技术服务有限公司 | Image similarity confirmation method and device |
| CN104392439B (en) * | 2014-11-13 | 2019-01-11 | 北京智谷睿拓技术服务有限公司 | The method and apparatus for determining image similarity |
| CN104504050A (en) * | 2014-12-20 | 2015-04-08 | 辽宁师范大学 | Retrieval method of color images based on quaternion invariants |
| CN104504049A (en) * | 2014-12-20 | 2015-04-08 | 辽宁师范大学 | Retrieval method of color images based on quaternion Harmonic-Fourier moments |
| CN105844283A (en) * | 2015-01-16 | 2016-08-10 | 阿里巴巴集团控股有限公司 | Method for identifying category of image, image search method and image search device |
| CN105844283B (en) * | 2015-01-16 | 2019-06-07 | 阿里巴巴集团控股有限公司 | Method, image search method and the device of image classification ownership for identification |
| CN106203461A (en) * | 2015-05-07 | 2016-12-07 | 中国移动通信集团公司 | A kind of image processing method and device |
| CN106557526A (en) * | 2015-09-30 | 2017-04-05 | 富士通株式会社 | The apparatus and method for processing image |
| CN106557526B (en) * | 2015-09-30 | 2020-06-30 | 富士通株式会社 | Apparatus and method for processing image |
| CN105589929A (en) * | 2015-12-09 | 2016-05-18 | 东方网力科技股份有限公司 | Image retrieval method and device |
| CN105589929B (en) * | 2015-12-09 | 2019-05-10 | 东方网力科技股份有限公司 | Image search method and device |
| CN105631037B (en) * | 2015-12-31 | 2019-02-22 | 北京恒冠网络数据处理有限公司 | A kind of image search method |
| CN105631037A (en) * | 2015-12-31 | 2016-06-01 | 北京恒冠网络数据处理有限公司 | Image retrieval method |
| CN105975497A (en) * | 2016-04-27 | 2016-09-28 | 清华大学 | Automatic microblog topic recommendation method and device |
| CN106056159A (en) * | 2016-06-03 | 2016-10-26 | 西安电子科技大学 | Image fine classification method based on Fisher Vector |
| CN106056159B (en) * | 2016-06-03 | 2019-03-26 | 西安电子科技大学 | Image sophisticated category method based on Fisher Vector |
| CN106203537B (en) * | 2016-08-31 | 2019-06-04 | 成都铁安科技有限责任公司 | A kind of current collecting bow lifting condition detection method and device |
| CN106203537A (en) * | 2016-08-31 | 2016-12-07 | 成都铁安科技有限责任公司 | A kind of current collecting bow lifting condition detection method and device |
| CN106651935A (en) * | 2016-11-29 | 2017-05-10 | 河南科技大学 | Multi-scale sampling-based texture image representation method |
| CN106651935B (en) * | 2016-11-29 | 2019-06-14 | 河南科技大学 | A kind of texture image representation method based on multi-scale sampling |
| CN108805148A (en) * | 2017-04-28 | 2018-11-13 | 富士通株式会社 | Handle the method for image and the device for handling image |
| CN109033308A (en) * | 2018-07-16 | 2018-12-18 | 安徽江淮汽车集团股份有限公司 | A kind of image search method and device |
| CN109276243A (en) * | 2018-08-31 | 2019-01-29 | 易念科技（深圳）有限公司 | Brain electricity psychological test method and terminal device |
| CN109446356A (en) * | 2018-09-21 | 2019-03-08 | 深圳市九洲电器有限公司 | A kind of multimedia document retrieval method and device |
| CN109271533A (en) * | 2018-09-21 | 2019-01-25 | 深圳市九洲电器有限公司 | A kind of multimedia document retrieval method |
| WO2020057347A1 (en) * | 2018-09-21 | 2020-03-26 | 深圳市九洲电器有限公司 | Multimedia file retrieval method and apparatus |
| CN109816025A (en) * | 2019-01-29 | 2019-05-28 | 宝鸡文理学院 | A kind of image search method based on image classification |
| CN110069654A (en) * | 2019-03-15 | 2019-07-30 | 平安城市建设科技(深圳)有限公司 | Source of houses searching method, device, equipment and computer readable storage medium |
| CN111339343A (en) * | 2020-02-12 | 2020-06-26 | 腾讯科技（深圳）有限公司 | Image retrieval method, device, storage medium and equipment |
| CN111353011A (en) * | 2020-02-27 | 2020-06-30 | 北京市商汤科技开发有限公司 | Location data set, building method and device thereof, and data processing method and device |
| CN111353011B (en) * | 2020-02-27 | 2024-05-17 | 北京市商汤科技开发有限公司 | Site data set, establishing method and device thereof, and data processing method and device |
| CN113868440A (en) * | 2020-06-30 | 2021-12-31 | 华为技术有限公司 | Method, device, equipment and medium for managing feature library |
| CN113868440B (en) * | 2020-06-30 | 2023-06-27 | 华为技术有限公司 | Feature library management method, device, equipment and medium |
| WO2022166604A1 (en) * | 2021-02-04 | 2022-08-11 | 腾讯科技（深圳）有限公司 | Image processing method and apparatus, computer device, storage medium, and program product |
| CN114611400A (en) * | 2022-03-18 | 2022-06-10 | 河北金锁安防工程股份有限公司 | Early warning information screening method and system |
| CN114611400B (en) * | 2022-03-18 | 2023-08-29 | 河北金锁安防工程股份有限公司 | Early warning information screening method and system |
| CN114926471A (en) * | 2022-05-24 | 2022-08-19 | 北京医准智能科技有限公司 | Image segmentation method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102402621A (en) | Image retrieval method based on image classification | |
| Mehmood et al. | Content-based image retrieval and semantic automatic image annotation based on the weighted average of triangular histograms using support vector machine | |
| Jain et al. | Content based image retrieval | |
| Penatti et al. | Comparative study of global color and texture descriptors for web image retrieval | |
| Li et al. | Free-hand sketch recognition by multi-kernel feature learning | |
| Saavedra et al. | Sketch based Image Retrieval using Learned KeyShapes (LKS). | |
| Kapoor et al. | Gaussian processes for object categorization | |
| Tabia et al. | Covariance-based descriptors for efficient 3D shape matching, retrieval, and classification | |
| Silva et al. | Graph-based bag-of-words for classification | |
| Ali et al. | A hybrid geometric spatial image representation for scene classification | |
| Leng et al. | 3D object understanding with 3D convolutional neural networks | |
| Laga | Semantics-driven approach for automatic selection of best views of 3D shapes | |
| Zhou et al. | Image retrieval based on effective feature extraction and diffusion process | |
| Wang et al. | View-based 3D object retrieval with discriminative views | |
| Liu et al. | Upright orientation of 3D shapes with convolutional networks | |
| Cao et al. | Hand posture recognition based on heterogeneous features fusion of multiple kernels learning | |
| Raikar et al. | Efficiency comparison of supervised and unsupervised classifier on content based classification using shape, color, texture | |
| Sethulekshmi et al. | Ayurvedic leaf recognition for plant classification | |
| Villamizar et al. | Online learning and detection of faces with low human supervision | |
| Karmakar et al. | An enhancement to the spatial pyramid matching for image classification and retrieval | |
| Wang et al. | Adaptively feature matching via joint transformational-spatial clustering | |
| Yang et al. | mPadal: a joint local-and-global multi-view feature selection method for activity recognition | |
| Jammalamadaka et al. | Human pose search using deep poselets | |
| Xiao et al. | Fast view-based 3D model retrieval via unsupervised multiple feature fusion and online projection learning | |
| Merino et al. | 2D image features detector and descriptor selection expert system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20120404 |