CN106649624B - Local feature point verification method based on global relationship consistency constraint - Google Patents

Local feature point verification method based on global relationship consistency constraint Download PDF

Info

Publication number
CN106649624B
CN106649624B CN201611109737.2A CN201611109737A CN106649624B CN 106649624 B CN106649624 B CN 106649624B CN 201611109737 A CN201611109737 A CN 201611109737A CN 106649624 B CN106649624 B CN 106649624B
Authority
CN
China
Prior art keywords
feature
points
image
point
feature points
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201611109737.2A
Other languages
Chinese (zh)
Other versions
CN106649624A (en
Inventor
姚金良
杨醒龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Electronic Science and Technology University
Original Assignee
Hangzhou Electronic Science and Technology University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Electronic Science and Technology University filed Critical Hangzhou Electronic Science and Technology University
Priority to CN201611109737.2A priority Critical patent/CN106649624B/en
Publication of CN106649624A publication Critical patent/CN106649624A/en
Application granted granted Critical
Publication of CN106649624B publication Critical patent/CN106649624B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Library & Information Science (AREA)
  • Databases & Information Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Multimedia (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a local feature point verification method based on global relationship consistency constraint. The invention comprises three parts: offline learning, local feature point quantization and feature point voting verification. Offline learning is used for the construction of visual vocabulary dictionaries. The local characteristic point quantization comprises three steps: 1. and extracting local characteristic points. 2. And (5) quantization of the feature descriptors. 3. Quantizing the main direction, the scale and the azimuth; the method used by the visual vocabulary verification part comprises two methods, namely weak relation consistency verification and strong geometry verification. The two methods adopt a voting mechanism to verify candidate feature points, and the steps are similar: 1. and acquiring candidate images and candidate feature points. 2. And verifying the candidate feature points through voting. The invention can adapt to the influence caused by image cutting, rotation, scale scaling and other transformations, and can be used in image retrieval and classification based on visual vocabularies.

Description

Local feature point verification method based on global relationship consistency constraint
Technical Field
The invention belongs to the field of computer image processing and machine vision, and relates to two kinds of local feature point verification based on visual vocabularies.
Background
Image retrieval based on local features is a mainstream copy image retrieval mode at present, local feature point descriptors are quantized into visual words, and a bag-of-words model is adopted to represent images, so that the method is an important method for current image retrieval. However, the visual vocabulary obtained by the descriptor quantization of the local features has no definite meaning with respect to the vocabulary in the natural language, and is easily affected by noise. In order to ensure the distinguishing capability of the visual words, the number of the visual words in the dictionary is more and better. But more visual vocabularies result in a weaker noise immunity and require more computational effort when the local features are quantized into visual vocabularies. Some researchers pay attention to the problem of ambiguity caused by quantizing local features into visual words, and partial solutions are proposed.
In order to increase the accuracy of local feature matching and reduce the ambiguity phenomenon of visual vocabularies, researchers are focusing on increasing the information content of the visual vocabularies to improve the recognition degree of the visual vocabularies. The method that multiple dictionaries are used for quantizing local features can improve the recall rate of image retrieval, but quantization of multiple dictionaries has intersection, and the intersection belongs to redundant data. The large amount of redundant data not only does not contribute to the improvement of the retrieval effect, but also affects the retrieval efficiency. In order to solve the problem, Zheng provides a bayesian combination method based on a plurality of dictionaries, which is used for reducing the association degree of the dictionaries and reducing redundant data. Mortensen adds a global texture vector to each local feature point to make the local feature have global properties. The method can be used for improving the distinguishing capability of visual vocabularies, but the method is not robust enough on the scale transformation of the image.
Researchers wish to improve the descriptive power of visual words by modeling their spatial dependencies (local features). Yao provides a method for generating a context descriptor of visual vocabularies, which considers the relationship between feature points and adjacent feature points and calculates the context similarity relationship between the visual vocabularies in the process of image detection to judge whether the visual vocabularies are correctly matched. Liu proposes two methods based on local features, and codes by taking the one-to-one and one-to-many relations between local feature points as feature points. The two methods belong to a method of prior verification, a good relation is established for the characteristic points before image detection, and the method has obvious advantages in retrieval speed. By adopting a similar method based on the neighbor local features, the global relationship between the feature points is often ignored.
In order to achieve global performance, researchers have considered global constraints on local features in recent years. Zheng finds that, in the image retrieval process, the query image and the correctly matched visual phrases in the candidate images have a relatively stable position relationship, for example, coordinates of the matched visual phrases are subtracted, and correctly matched points fall into a relatively concentrated position. Zhou uses a compact spatial coding method to describe the mutual position relationship of visual vocabularies. However, this method is not ideal for supporting the rotation transformation of the image, and it is necessary to construct positional relationships in multiple directions to improve the robustness of the rotation transformation.
Aiming at the problem of low matching accuracy caused by the phenomenon of ambiguity caused after the local features are quantized into the visual vocabularies, the two methods of the invention provide that the global relationship based on the local features is utilized to enhance the visual vocabulary distinguishing capability. The method meets the requirements of compactness and robustness, can be used for various edits and transformations of the image, and has a good effect;
disclosure of Invention
The invention aims to provide two local feature point verification methods based on global relationship consistency constraint aiming at the defects of the prior art.
The technical scheme adopted by the invention for solving the technical problems is as follows:
1. the local feature point verification method based on the global relationship consistency constraint is characterized by comprising the following three parts: (1) an off-line learning part, (2) a characteristic point quantization part, and (3) a characteristic point voting verification part; the off-line learning part is used for constructing a visual vocabulary dictionary; the characteristic point quantization part quantizes local characteristics according to a visual vocabulary dictionary obtained by off-line learning; the feature point voting verification part is used for verifying the feature points in the candidate images, and is specifically realized as follows:
and (1) performing off-line learning, and grouping and clustering a large number of samples to obtain a visual vocabulary dictionary.
And (2) quantizing the feature points of the query image through a visual vocabulary dictionary to obtain visual vocabularies.
And (3) matching the visual words of the query image in an index database to obtain candidate feature points, and establishing a relationship by using the unique image identification of the candidate feature points to obtain a plurality of candidate images.
And (4) verifying the feature points through the constraint of the global consistency weak relation or the strong geometric relation, and finally achieving the aim of verifying the candidate images.
(1) The off-line learning part is characterized by constructing a visual vocabulary dictionary of local features, and comprises the following specific steps:
1-1, selecting a large number of images to construct an image library, extracting local feature points and feature descriptors of the images in the image library, and constructing the extracted feature descriptors into a sample library;
1-2, obtaining a visual vocabulary dictionary through a sample library; specifically, feature vectors of feature descriptors in a sample library are grouped, K class centers are obtained on each feature group through K mean value clustering, each class center is a feature vector and represents a root word in a visual vocabulary, and the K class centers are root word sets of the feature groups; combining the root sets constructed on each feature group to obtain a visual vocabulary dictionary;
(2) the characteristic point quantization part comprises two parts: quantization of local feature descriptors, principal direction, scale and coordinate quantization.
2-1. local feature descriptor quantization: extracting local feature point set S ═ { P ═ P for input imagei,i∈[0,Q]Q is the number of local feature points in the input image, PiRefers to the ith local feature point; and the local characteristic point P is quantized by grouping according to the visual vocabulary dictionaryiIs quantized into the visual vocabulary VWi(ii) a The method comprises the following specific steps:
2-1-1, extracting local characteristic point P from input imageiCharacteristic descriptor F ofiPosition (Px)i,Pyi) Dimension σiAnd a main direction thetaiInformation, i.e. local characteristic points PiIs represented by [ Fi,θi,σi,Pxi,Pyi]；
2-1-2. for each local feature point PiCharacteristic descriptor F ofiObtaining visual vocabularies by adopting a grouping quantization method according to the visual vocabulary dictionary; the grouping quantization based on visual vocabulary dictionary is to divide the feature descriptor FiDividing into M groups, each group is D/M characteristics, wherein D is a characteristic descriptor FiThe dimension of the feature vector; then, the feature vectors of each group are independently quantized into V according to the visual vocabulary dictionary trained in the step 1-2jThen use grouping quantization to obtain feature descriptor FiVisual vocabulary VWiComprises the following steps:
Figure BDA0001172226200000041
wherein, L is the number of the corresponding group of the word roots in the visual vocabulary dictionary; thereby a local feature point PiIs denoted as [ VWi,θi,σi,Pxi,Pyi](ii) a The quantization of each group of feature vectors is realized by searching the nearest class center in the root set of the group based on Euclidean distance and taking the subscript of the class center as the quantization result;
and 2-2. quantizing the main direction, the scale and the coordinates: the principal direction θ mentioned above is a floating point type of camber value, here quantized to an integer type angle value θ: thetai＝θi*180/π。
Same location information (Px)i,Pyi) And the scale σiBut also quantized into a reshape. In quantizing the scale, aiMultiplying by 100 and then rounding preserves some precision.
(3) The feature point voting verification part is realized in two ways: the first is a local feature point verification method based on global weak relation consistency constraint, and the second is a local feature point verification method based on global strong geometric relation consistency constraint. The voting mechanism is adopted in both modes: in a candidate image, if a certain candidate feature point and a certain proportion of context feature points simultaneously satisfy the constraint relation, the point is considered as a correctly matched feature point.
In the process of feature point verification, whether feature points are matched or not is verified mainly according to the azimuth feature, the principal direction feature and the scale feature of the local feature points; two ways are presented below:
3-1, verifying consistency of weak relationship: the method is based on the principle that the main direction and the scale size between the correct matching feature points have consistency. The method comprises the following specific steps:
and 3-1-1, obtaining a large number of candidate feature points by matching visual words obtained by quantizing the feature points of the query image with the feature points in the index database. And (5) constructing a Hash table by using the image IDs (imgIds) of the feature points as key words, and finding a plurality of candidate images.
3-1-2. in the query image, if the scale of a feature point is smaller than a certain context feature point, in the candidate image, the corresponding candidate feature point should also satisfy the constraint. At the same time, the principal direction should also satisfy this condition, and the formula is as follows:
Figure BDA0001172226200000051
wherein Scl represents scale, Ori represents principal direction, and subscripts ai, aj represents ith, j characteristic point of a image. Mi,jA value of 1 indicates that the two feature points i, j satisfy the consistency relationship constraint, and the number of votes of the feature point j to the feature point i is increased by 1. If a certain feature point and a certain proportion of context feature points satisfy the constraint, the candidate feature point is considered as a correct matching point. In the process of image retrieval, the voting sum S obtained by correctly matching the feature points is calculated.
Figure BDA0001172226200000052
Wherein the content of the first and second substances,
Figure BDA0001172226200000053
the number of votes obtained at the feature point i, and Th is a voting threshold. The vote sum S of the candidate images is used for measuring the similarity degree of one candidate image.
3-2, strong geometric verification: the strong geometric verification is to calculate two angular relationships of main direction difference and angular position difference between characteristic points. In the two angles used in the method, the principal direction difference is the difference in principal direction between the two feature points. The principal direction difference can be calculated by the following formula:
β＝|Orii-Orij|
Oriiis the i principal direction, Ori, of the point to be verifiedjIs the main direction of the context feature point j of the point i to be verified.
The angular difference can be calculated by the following formula:
Figure BDA0001172226200000054
wherein, arctan2 (P)i,Pj) Is two characteristic points Pi,PjThe angle between the connecting line and the horizontal direction. Strong tableThe relationship verification method comprises the following steps:
and 3-2-1, obtaining a large number of matched feature points by matching the visual vocabulary obtained by quantizing the feature points of the query image with the feature points in the index database. And (5) constructing a Hash table by using the image IDs (imgIds) of the feature points as key words, and finding a plurality of candidate images.
3-2-2, between the feature points which are matched correctly, the main direction difference between two feature points of the query image and the main direction difference between two feature points corresponding to the candidate image are approximately equal. The same should also be the case for angular differences that are approximately equal, corresponding to the difference in principal direction differences between feature points:
Figure BDA0001172226200000061
wherein the content of the first and second substances,
Figure BDA0001172226200000062
representing the principal direction difference of the feature points i and j in image a,
Figure BDA0001172226200000063
indicating the principal direction difference of the corresponding feature point in image b.
And difference in angular head:
Figure BDA0001172226200000064
wherein the content of the first and second substances,
Figure BDA0001172226200000065
representing the angular difference of the feature points i and j in image a,
Figure BDA0001172226200000066
indicating the angular position difference of the corresponding feature point in image b.
Figure BDA0001172226200000067
If the two points i and j satisfy the constraint relation, Mi,jEqual to 1. Here, whether the candidate feature point is a correctly matched feature point is determined by voting. If a certain feature point and a certain proportion of context feature points in the image satisfy the relationship, the feature point is considered as a correctly matched feature point. And counting correct matching points in each candidate image, and calculating a voting sum S obtained by correctly matching the feature points.
Figure BDA0001172226200000068
Wherein the content of the first and second substances,
Figure BDA0001172226200000069
is the number of votes obtained for feature point i. The vote sum S of the candidate images is used for measuring the similarity degree of one candidate image.
Compared with the prior art, the invention has the following beneficial effects:
the method can be used for large-scale image retrieval, and the retrieval efficiency and accuracy are improved; in addition, the method adopts the global relation of the local feature points to verify the candidate image, so when the candidate feature points of the candidate image are too many, the feature points with larger scale in the candidate image can be adopted, and the experimental result proves that the effect is better and the speed is higher than that of using all the candidate feature points. Meanwhile, the method has good robustness on image transformation such as scaling, rotation, cutting and the like of the image.
Drawings
FIG. 1 shows a flow chart of the present invention;
FIG. 2 is a diagram of weak relationship consistency verification;
FIG. 3 is a schematic illustration of strong geometry verification;
Detailed Description
In the implementation, the algorithm uses a SIFT operator, and in the following description, the descriptor of the local feature point refers to SIFT and is not specifically indicated. The first method is a local feature point verification method based on global weak relation consistency constraint, and the second method is a local feature point verification method based on global strong geometric relation consistency constraint. The method can be used in image retrieval and image identification and detection methods based on local feature points.
Embodiments of the present invention will be further described with reference to the accompanying drawings.
Fig. 1 is a flow chart of the present invention, showing the relationship of each part and the flow thereof. The two methods specifically comprise the following steps: an off-line learning section, a feature point quantizing section, and a feature point vote verifying section. The off-line learning part is used for constructing a visual vocabulary dictionary and comprises the steps of obtaining sample points, clustering the sample points and generating the dictionary. The characteristic point quantization comprises three parts, namely local characteristic point extraction, main direction, direction and scale quantization and descriptor quantization. The third part of feature point voting verification comprises two parts, wherein candidate images and candidate feature points are found in an index library, and the candidate feature points are verified in a voting mode.
(1) The offline learning part in fig. 1 mainly includes: and (5) constructing a visual vocabulary dictionary.
And selecting a large number of images, and extracting characteristic points of the images to be used as a sample learning library. Then, grouping the feature vectors of the feature descriptors in the feature descriptor sample library; obtaining K class centers on each feature group through K mean clustering, wherein each class center is a feature vector which represents a root in a visual vocabulary, and the K class centers are root sets of the feature group; selecting a root word from the root set for each feature set generates a visual vocabulary. And combining the root sets constructed on each feature group to obtain a visual vocabulary dictionary. In this embodiment, the feature descriptors of the local feature points are divided into 4 groups, each group has 8 feature values, 64 class centers are constructed by K-means clustering, and each class center is the root of the group; 4 class centers are the visual dictionary of the method; the class centers of each group are stored in an array and stored in a file; the method requires loading the array into memory when performing the quantized local feature descriptors.
And finally, obtaining the visual vocabulary by the local feature descriptors in the sample library according to the visual vocabulary dictionary by adopting a grouping quantization method. In the present embodiment, the extracted feature descriptors of SIFT are 32-dimensional feature vectors.
(2) The specific steps of the characteristic point quantization part in fig. 1 are as follows:
firstly, extracting a local feature point set S ═ P from an imagei,i∈[0,Q]Q is the number of local characteristic points in the image; and the local characteristic point P is quantized by grouping according to a visual vocabulary dictionaryiIs quantized into the visual vocabulary VWi. The method comprises the following specific steps:
in the local feature point extraction in fig. 1, in this embodiment, the detected local feature point is described by using a SIFT descriptor; a local feature point (P)i) By SIFT descriptor can be expressed as: [ F ]i,θi,σi,Pxi,Pyi](ii) a Wherein FiRepresenting the feature descriptor vector by using a gradient histogram; thetaiIs a main direction; sigmaiIs the scale of the local feature point, (Px)i,Pyi) Is a local feature point (P)i) Spatial position in the image. In this embodiment, FiIs set as a 32-dimensional feature vector. Through local feature point extraction and description, an image is represented as a set of SIFT descriptors.
The feature descriptor in fig. 1 quantifies the feature descriptor for each local feature point (F)i) Obtaining visual vocabularies by adopting a grouping quantization method according to the visual vocabularies; grouping quantization is to sub-divide the feature descriptor Fi(D-dimensional feature vectors) are divided into M groups of D/M features, and then the feature vectors of each group are quantized into V individually according to a trained dictionaryjThen, the visual Vocabulary (VW) for obtaining the feature descriptor by using the grouping quantization is:
Figure BDA0001172226200000081
wherein L is the number of the corresponding group of the word roots in the visual vocabulary dictionary. A local feature point P is quantized by the local feature descriptoriIs denoted as [ VWi,θi,σi,Pxi,Pyi]。
FIG. 1 shows 22 for quantizing other SIFT feature attributes, where the attribute to be quantized includes the principal direction θiDimension σiAnd orientation (Px)i,Pyi). Principal direction thetaiIs a floating point type of camber value, quantized here to an integer angular value θi＝θi*180/π；
Same location information (Px)i,Pyi) And scale is also quantized to shape. In quantizing the scale, 100 is multiplied and then rounded, and certain precision is reserved.
(3) The feature point voting verification section in fig. 1 mainly includes 31 acquiring candidate images and candidate feature points, and 32 verifying the candidate feature points by voting. In the image retrieval application, a large number of candidate feature points are obtained in an index database according to visual words of feature points of an inquired image, and a plurality of candidate images are obtained by the candidate feature points according to an image ID (imgId). The two methods verify candidate characteristic points in the candidate image through global consistency constraint to achieve the aim of image copy detection. The verification of the feature points in the invention comprises two methods, wherein one method is to verify candidate feature points by correctly matching the relationship consistency between the main direction and the scale relative size between the feature points, so as to become weak relationship consistency verification. The other is verification of feature points through two strong geometric relations of principal direction difference and angular position difference between the feature points, which is called strong geometric verification.
3-1, verifying weak consistency relationship: and verifying whether the feature points are correctly matched feature points or not by adopting a voting mode according to the principle that the main direction and the scale size between the correctly matched feature points are consistent. The method comprises the following specific steps:
and 3-1-1, obtaining a large number of matched feature points by matching the visual vocabulary obtained by quantizing the feature points of the query image with the feature points in the index database. And (5) constructing a Hash table by using the image IDs (imgIds) of the feature points as key words, and finding a plurality of candidate images.
3-1-2. As shown in FIGS. 2(a) and 2(b), in the query image, if the scale of one feature point is smaller than that of another, then the same visual vocabulary should satisfy the constraint in the candidate image. While the main direction should also fulfil this condition. If a feature point and a certain proportion of feature points in the candidate image satisfy the relationship, the feature point is considered as a correctly matched feature point. The following two equations are shown:
Figure BDA0001172226200000101
wherein Scl represents scale, Ori represents principal direction, and subscripts ai, aj represents ith, j characteristic point of a image. Mi,jA value of 1 indicates that the two feature points i, j satisfy the consistency relationship constraint, and the number of votes of the feature point j to the feature point i is increased by 1. If a certain feature point and a certain proportion of contextual feature points satisfy the constraint, the point is considered to be the correct point. In the process of image retrieval, the voting sum S obtained by correctly matching the feature points is calculated.
Figure BDA0001172226200000102
Wherein the content of the first and second substances,
Figure BDA0001172226200000103
the number of votes obtained at the feature point i, and Th is a voting threshold. The sum of votes for each candidate image is used to measure the similarity of a candidate image.
The pseudo code of the specific execution process is as follows:
Figure BDA0001172226200000104
3-2, strong geometric verification: the strong geometric verification is to calculate two angular relationships of main direction difference and angular position difference between characteristic points. The attributes of the feature points that need to be used include principal direction and orientation features. In the two angles used by the method, the main direction difference is the difference of the main directions between the two characteristic points, and the angle difference is the included angle between the main direction and the connecting line between the two characteristic points. The principal direction difference can be calculated by the following formula:
β＝|Orii-Orij|
the angular difference can be calculated by the following formula:
Figure BDA0001172226200000111
and 3-2-1. similar to the step 3-1-1, a visual vocabulary is obtained by quantizing the characteristic points of the query image, and the visual vocabulary is matched with the characteristic points in the index database to obtain a large number of matched characteristic points. And (5) constructing a Hash table by using the image IDs (imgIds) of the feature points as key words, and finding a plurality of candidate images.
3-2-2. between the correctly matched feature points, the main direction difference of the two feature points of the inquired image and the main direction difference of the two feature points corresponding to the candidate image should be approximately equal. The same should also be the case for angular differences that are approximately equal, corresponding to the difference in principal direction differences between feature points:
and difference in angular head
Should approach 0.
Figure BDA0001172226200000114
If the two points i and j satisfy the constraint relation, Mi,jEqual to 1. Here, whether the candidate feature point is a correctly matched feature point is determined by voting. If a certain feature point and other candidate feature points in the image in a certain proportion satisfy the relationship, the feature point is considered as a correctly matched feature point. And counting correct matching points in each candidate image, and calculating the voting sum S obtained by the correct matching feature points.
Figure BDA0001172226200000115
Wherein the content of the first and second substances,
Figure BDA0001172226200000121
is the number of votes obtained for feature point i. The sum of votes for each candidate image is used to measure the similarity of a candidate image.
The pseudo code of the specific execution process is as follows:
Figure BDA0001172226200000122
having described embodiments of the invention in detail, it will be appreciated that variations and modifications can be effected within the scope of the invention as described above and as particularly set forth in the claims by a person of ordinary skill in the art without departing from the scope of the invention.

Claims (3)

1. The local feature point verification method based on the global relationship consistency constraint is characterized by comprising the following three parts: (1) an off-line learning part, (2) a characteristic point quantization part, and (3) a characteristic point voting verification part; the off-line learning part is used for constructing a visual vocabulary dictionary; the characteristic point quantization part quantizes local characteristics according to a visual vocabulary dictionary obtained by off-line learning; the feature point voting verification part is used for verifying the feature points in the candidate images, and is specifically realized as follows:
step (1), off-line learning is carried out, and a visual vocabulary dictionary is obtained by grouping and clustering a large number of samples;
quantizing the feature points of the query image through a visual vocabulary dictionary to obtain visual vocabularies;
matching visual words of the query image in an index library to obtain candidate feature points, and establishing a relationship by using unique image identifiers of the candidate feature points to obtain a plurality of candidate images;
verifying the characteristic points through strong geometric relationship constraint to achieve the aim of verifying the candidate images;
step (4) verifying the feature points by adopting strong geometric relationship constraint; the method verifies the feature points according to the principle that the main direction difference and the angular position difference between the correctly matched feature points meet the consistency constraint principle; principal directionβ ═ Orii-Orij|；OriiIs the i principal direction, Ori, of the point to be verifiedjIs the principal direction of the contextual feature point j of the verification point i; the angular position difference is an included angle between a connecting line of the feature point to be verified and the upper and lower feature points thereof and the main direction of the feature point to be verified, and can be calculated by the following formula:
Figure FDA0002311019970000012
Figure FDA0002311019970000011
arctan2(Pi，Pj) For calculating characteristic points (P)i，Pj) The connecting line of the two points forms an included angle with the horizontal direction; the method comprises the following specific steps:
4-1, matching visual words obtained by quantizing the characteristic points of the query image with the characteristic points of the built index in the index database to obtain a large number of matched characteristic points; establishing a Hash table by taking the image IDs of the characteristic points as key words to find a plurality of candidate images;
4-2, because SIFT feature points have good rotation robustness, the main direction difference between two feature points of the query image and the main direction difference between two feature points corresponding to the candidate image are approximately equal; similarly, the angular differences should also be approximately equal; difference of principal direction difference between corresponding feature points: dif _ Orii，j＝|βaij-βbijAnd the difference in angular difference Dif _ Dir ═ αaij-αbijI should approach 0;
Figure FDA0002311019970000021
if the characteristic points i and j satisfy the constraint relationship, Mi，jEqual to 1; wherein the content of the first and second substances,
Figure FDA0002311019970000022
representing the angular difference of the feature points i and j in image a,
Figure FDA0002311019970000023
representing the angular position difference of the corresponding feature point in image b,
Figure FDA0002311019970000024
representing the principal direction difference of the feature points i and j in image a,
Figure FDA0002311019970000025
representing the principal direction difference of the corresponding feature point in the image b;
judging whether the candidate feature points are correctly matched feature points or not by adopting a voting mode, and calculating the voting sum S obtained by correctly matching the feature points;
wherein the content of the first and second substances,
Figure FDA0002311019970000027
the number of tickets obtained by the characteristic point i; and when selecting the result, selecting a plurality of images from large to small as copy images according to the voting sum of the candidate images.
2. The local feature point verification method based on global relationship consistency constraint according to claim 1, characterized in that the off-line learning part comprises the following specific implementation steps:
1-1, selecting a large number of images to construct an image library, extracting local feature points and feature descriptors of the images in the image library, and constructing the extracted feature descriptors into a sample library;
1-2, obtaining a visual vocabulary dictionary through a sample library; specifically, feature vectors of feature descriptors in a sample library are grouped, K class centers are obtained on each feature group through K mean value clustering, each class center is a feature vector and represents a root word in a visual vocabulary, and the K class centers are root word sets of the feature groups; and combining the root sets constructed on each feature group to obtain a visual vocabulary dictionary.
3. The local feature point verification method based on global relationship consistency constraint according to claim 2, wherein the quantization of the feature points in the feature point quantization part comprises two parts: quantizing local feature descriptors, and quantizing main directions, scales and coordinates;
2-1. local feature descriptor quantization: extracting local feature point set S ═ { P ═ P for input imagei，i∈[0，Q]Q is the number of local feature points in the input image, PiRefers to the ith local feature point; and the local characteristic point P is quantized by grouping according to the visual vocabulary dictionaryiIs quantized into the visual vocabulary VWi(ii) a The method comprises the following specific steps:
2-1-1, extracting local characteristic point P from input imageiCharacteristic descriptor F ofiPosition (Px)i，Pyi) Dimension σiAnd a main direction thetaiInformation, i.e. local characteristic points PiIs represented by [ Fi，θi，σi，Pxi，Pyi]；
2-1-2. for each local feature point PiCharacteristic descriptor F ofiObtaining visual vocabularies by adopting a grouping quantization method according to the visual vocabulary dictionary; the grouping quantization based on visual vocabulary dictionary is to divide the feature descriptor FiDividing into M groups, each group is D/M characteristics, wherein D is a characteristic descriptor FiThe dimension of the feature vector; then, the feature vectors of each group are independently quantized into V according to the visual vocabulary dictionary trained in the step 1-2jThen use grouping quantization to obtain feature descriptor FiVisual vocabulary VWiComprises the following steps:
Figure FDA0002311019970000031
Figure FDA0002311019970000032
wherein, L is the number of the corresponding group of the word roots in the visual vocabulary dictionary; thereby a local feature point PiIs represented as [ V ]Wi，θi，σi，Pxi，Pyi](ii) a The quantization of each group of feature vectors is realized by searching the nearest class center in the root set of the group based on Euclidean distance and taking the subscript of the class center as the quantization result;
and 2-2. quantizing the main direction, the scale and the coordinates: the principal direction θ mentioned above is a floating point type of camber value, here quantized to an integer type angle value θ: thetai＝θi*180/π；
Same location information (Px)i，Pyi) And the scale σiAlso quantized into a reshape; in quantizing the scale, aiMultiplying by 100 and then rounding preserves some precision.
CN201611109737.2A 2016-12-06 2016-12-06 Local feature point verification method based on global relationship consistency constraint Active CN106649624B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611109737.2A CN106649624B (en) 2016-12-06 2016-12-06 Local feature point verification method based on global relationship consistency constraint

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611109737.2A CN106649624B (en) 2016-12-06 2016-12-06 Local feature point verification method based on global relationship consistency constraint

Publications (2)

Publication Number Publication Date
CN106649624A CN106649624A (en) 2017-05-10
CN106649624B true CN106649624B (en) 2020-03-03

Family

ID=58818347

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611109737.2A Active CN106649624B (en) 2016-12-06 2016-12-06 Local feature point verification method based on global relationship consistency constraint

Country Status (1)

Country Link
CN (1) CN106649624B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104036012A (en) * 2014-06-24 2014-09-10 中国科学院计算技术研究所 Dictionary learning method, visual word bag characteristic extracting method and retrieval system
CN104573681A (en) * 2015-02-11 2015-04-29 成都果豆数字娱乐有限公司 Face recognition method
CN105678349A (en) * 2016-01-04 2016-06-15 杭州电子科技大学 Method for generating context descriptors of visual vocabulary

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104036012A (en) * 2014-06-24 2014-09-10 中国科学院计算技术研究所 Dictionary learning method, visual word bag characteristic extracting method and retrieval system
CN104573681A (en) * 2015-02-11 2015-04-29 成都果豆数字娱乐有限公司 Face recognition method
CN105678349A (en) * 2016-01-04 2016-06-15 杭州电子科技大学 Method for generating context descriptors of visual vocabulary

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Near-duplicate image retrieval based on contextual descriptor;Yao J L 等;《IEEE Signal Processing Letters》;20151231;第22卷(第9期);1404-1408 *
基于局部特征的视觉上下文分析及其应用;周文罡;《中国科学技术大学:信号与信息处理》;20110919;正文第4.3节 *

Also Published As

Publication number Publication date
CN106649624A (en) 2017-05-10

Similar Documents

Publication Publication Date Title
Gao et al. View-based 3D object retrieval: challenges and approaches
Leng et al. A 3D model recognition mechanism based on deep Boltzmann machines
CN102693311B (en) Target retrieval method based on group of randomized visual vocabularies and context semantic information
CN105224937B (en) Fine granularity semanteme color pedestrian recognition methods again based on human part position constraint
CN104615676B (en) One kind being based on the matched picture retrieval method of maximum similarity
Guo et al. 3D free form object recognition using rotational projection statistics
Yue et al. Robust loop closure detection based on bag of superpoints and graph verification
Liu et al. Uniting keypoints: Local visual information fusion for large-scale image search
Gao et al. Democratic diffusion aggregation for image retrieval
Fan et al. Satellite image matching method based on deep convolutional neural network
CN104199842A (en) Similar image retrieval method based on local feature neighborhood information
Furuya et al. Diffusion-on-manifold aggregation of local features for shape-based 3d model retrieval
Johns et al. From images to scenes: Compressing an image cluster into a single scene model for place recognition
JPWO2017006852A1 (en) Image collation apparatus, image collation method, and program
Tao et al. A pipeline for 3-D object recognition based on local shape description in cluttered scenes
CN102208033A (en) Data clustering-based robust scale invariant feature transform (SIFT) feature matching method
CN105760875A (en) Binary image feature similarity discrimination method based on random forest algorithm
CN105678349B (en) A kind of sub- generation method of the context-descriptive of visual vocabulary
CN111597367B (en) Three-dimensional model retrieval method based on view and hash algorithm
JP2018136946A (en) Three-dimensional cad model part search method and three-dimensional cad model search method
CN106649624B (en) Local feature point verification method based on global relationship consistency constraint
CN106991431B (en) Post-verification method for local feature point matching pairs
Altintakan et al. An improved BOW approach using fuzzy feature encoding and visual-word weighting
Lu et al. Efficient image categorization with sparse Fisher vector
CN108959650A (en) Image search method based on symbiosis SURF feature

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
EE01 Entry into force of recordation of patent licensing contract

Application publication date: 20170510

Assignee: Hangzhou Zihong Technology Co., Ltd

Assignor: Hangzhou University of Electronic Science and technology

Contract record no.: X2021330000654

Denomination of invention: Local feature point verification method based on global relationship consistency constraint

Granted publication date: 20200303

License type: Common License

Record date: 20211104

EE01 Entry into force of recordation of patent licensing contract