CN105045841B - With reference to gravity sensor and the characteristics of image querying method of image characteristic point angle - Google Patents

With reference to gravity sensor and the characteristics of image querying method of image characteristic point angle Download PDF

Info

Publication number
CN105045841B
CN105045841B CN201510379232.7A CN201510379232A CN105045841B CN 105045841 B CN105045841 B CN 105045841B CN 201510379232 A CN201510379232 A CN 201510379232A CN 105045841 B CN105045841 B CN 105045841B
Authority
CN
China
Prior art keywords
image
characteristic point
angle
characteristic
gravity sensor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510379232.7A
Other languages
Chinese (zh)
Other versions
CN105045841A (en
Inventor
陈靖
张运超
王涌天
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Institute of Technology BIT
Original Assignee
Beijing Institute of Technology BIT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Institute of Technology BIT filed Critical Beijing Institute of Technology BIT
Priority to CN201510379232.7A priority Critical patent/CN105045841B/en
Publication of CN105045841A publication Critical patent/CN105045841A/en
Application granted granted Critical
Publication of CN105045841B publication Critical patent/CN105045841B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Library & Information Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a kind of combination gravity sensor and the characteristics of image querying method of image characteristic point angle, using the camera acquisition target image for carrying gravity sensor, obtain the image pattern with gravitation information, it is handled as follows for query image and image pattern, the overall compact vector for forming image pattern is characterized：Extract the characteristic point comprising gravitation information of image；Angular nature value in all characteristic points that will be extracted all increases an angle calculated by formula and is rotated, and all of image characteristic point carries out bag of words training in postrotational image, obtains multiple cluster centres；Feature coding is carried out to all of image characteristic point in postrotational image, characteristic point is mapped on each cluster centre, and subregion is carried out according to characteristic point angle, characteristic aggregation is carried out on each subregion, formed overall compact vector and characterize.Method, the image pattern that lookup is most matched with query image are searched with arest neighbors.

Description

With reference to gravity sensor and the characteristics of image querying method of image characteristic point angle
Technical field
The invention belongs to field of computer technology, it is related to the part of a kind of combination gravity sensor and image characteristic point angle Characteristic aggregation method.
Background technology
Local feature is polymerized the important research content as current massive image retrieval, is that a kind of image to after coding is special The technology for a little carrying out polymeric compressing is levied, characteristic aggregation is carried out by by multiple characteristic points of an image, whole image can be used Single vector-quantities carry out compact sign.The massive image retrieval algorithm of current main-stream is all calculated using bag of words and its mutation Method, will image characteristic point by feature coding vision word is mapped to characterize, the acquisition of vision word generally uses K averages The methods such as cluster are obtained, and the general token of image will then be mapped to the characteristic point of vision word using local feature polymerization technique Carry out accumulation polymerization.The quality of local feature polymerization, directly affects image retrieval precision.
Traditional local feature polymerization technique is mostly by coded quantization to same vision word or cluster centre Feature simply sue for peace or be most worth, and the characteristic point for participating in polymerization is only required to feature point description symbol vector in theorem in Euclid space Relatively current cluster centre.Such method relies primarily on the raising of cluster centre number to lift image retrieval precision, Increase the dimension of visual dictionary.But the lifting of cluster centre number sacrifices training time, feature coding time and figure The nearest _neighbor retrieval time as between, and for visual dictionary higher, many correct match points may be assigned to not On same cluster centre, and then influence retrieval precision.
In recent years, as both at home and abroad for the further investigation of local feature polymerization, Jegou et al. proposes Hamming insertion Concept (Jegou, H., M.Douze, et al. (2008) .Hamming embedding and weak geometric consistency for large scale image search.Computer Vision–ECCV 2008,Springer: 304-317.), the characteristic point fallen after coding in same vision word is carried out into hamming code, after and if only if hamming code Distance is just considered a pair of match points less than the point of a certain threshold value, that is to say in same vision word enterprising step refining feature The separating capacity of point.Compared to simple increase dictionary dimension, the mode of Hamming insertion has very for retrieval precision and speed Big lifting.Based on this thought, Zhao et al. (Zhao W L, J é gou H, Gravier G.Oriented pooling for dense and non-dense rotation-invariant features[C]//BMVC-24th British Machine Vision Conference.2013.) propose that characteristic point is carried out into subregion in same vision word according to characteristic point angle gathers Close.Rotation yet with image can cause not fixed according to the partition number of characteristic point angular divisions, it is impossible to directly carry out most Neighbor searching.Zhao proposes the rotation that may occur using the method exhaustion image of traversal, the basis in arest neighbors search procedure What two images may occur rotates against, and calculates the distance under every kind of rotational case, takes apart from minimum value as two images The distance between.On the one hand, the method estimates the corresponding situation of angle subregion, greatly according to rotating against that image may occur The big amount of calculation that increased in arest neighbors search procedure.On the other hand can not possibly be by exhaustion for the actual rotation that image occurs Algorithm is accurately estimated.In addition, the method taken apart from minimum value can reduce the punishment to dissimilar image, and then influence retrieval essence Degree.
The content of the invention
In view of this, the invention provides a kind of combination gravity sensor and the characteristic aggregation side of image characteristic point angle Method, increases the differentiation power of characterization image, improves image retrieval precision.
In order to achieve the above object, technical scheme comprises the following steps：
Step 1：Using the camera acquisition target image for carrying gravity sensor, the image with gravitation information is obtained Sample, is stored in database；Three reference axis x-axis, y-axis, z-axis of camera coordinate system when gravitation information is specially shooting image It is respectively relative to the angle with gravity direction；Wherein z-axis is the axle perpendicular to video camera imaging face, and the plane that x, y-axis are formed is It is video camera imaging face.
Y-axis positive direction is placed in it is consistent with gravity direction, will now camera position as reference position, at reference position Gravitation information is [gx(0),gy(0),gz(0) it is], 0 with image rotation angle during reference position.
Step 2：The behaviour for carrying out following steps 201~204 as image i using each width image pattern in database successively Make, the overall compact vector for forming all image patterns of correspondence is characterized.
Step 201, for image i, characteristic point is extracted to image pattern using feature extraction algorithm, comprising figure in characteristic point As the gravitation information of i is [gx(i),gy(i),gz(i)]。
Step 202, the gravity direction of image i turn clockwise angle [alpha] in camera coordinates systemiFor：
The angular nature value in all characteristic points in the image i that will be extracted all increases αi, obtain postrotational image i’。
Step 203, all of image characteristic point in image i ' is carried out bag of words training, obtain K cluster centre.
Step 204, feature coding is carried out to all of image characteristic point in image i ', characteristic point is mapped to each cluster On the region of the specified size where center, then the characteristic point fallen on each cluster centre is carried out according to characteristic point angle Subregion, carries out characteristic aggregation on each subregion, and the overall compact vector for forming image pattern i ' is characterized.
Step 3：Using query image as image i, the operation of above-mentioned steps 201~204 is performed, form correspondence query image Overall compact vector characterize；Method is searched using arest neighbors, is searched with the query image most in image pattern in database The image pattern matched somebody with somebody.
Further, when using camera acquisition target image, from different far and near scalings, different rotary angle and not It is acquired with the angle of target side.
Further, in step 204, carry out being calculated using partial polymerization descriptor VLAD during characteristic aggregation on each subregion Method.
Beneficial effect：
1st, the present invention proposes a kind of method that utilization gravity sensor accurately estimates image rotation, is calculated using gravitation information Image rotation angle, by all image rotations to horizontal direction, the image characteristic point of its matching has the characteristic of identical angle, Subregion is carried out according to angle in same vision word, the differentiation power of characterization image is increased, image retrieval precision is improve
2nd, on the one hand the present invention have compressed feature relative to the simple method for lifting retrieval precision by increase characteristic dimension When coding spends, the distinction between another aspect characteristic point is guaranteed.
3rd, the present invention also has the quick characteristic of image retrieval, more conventional vision word partition method, without in retrieval Exhaustive subregion corresponding relation in achievement, subregion correspondence lookup in rear end directly is changed into front end in the course of the polymerization process will matching spy Levy a rotation to same angle.Greatly reduce the image retrieval time.
Brief description of the drawings
Fig. 1 is the exemplary process diagram of this method embodiment.
Fig. 2 is the schematic diagram of the calculating image rotation angle of this method embodiment step 2.
Specific embodiment
Develop simultaneously embodiment below in conjunction with the accompanying drawings, and the present invention will be described in detail.
Embodiment 1, in embodiments of the present invention, the system embodiment devises that a kind of to combine gravity sensor special with image Levy the local feature polymerization of an angle.It is poly- that the technical scheme can carry out classification according to the principal direction angle of image characteristic point Close, the overall characterization vector of image can be made with more discrimination, be conducive to improving image retrieval precision.By intelligent terminal etc. The acceleration of gravity of equipment record image can accurately calculate the attitude rotation of image, and then by image integral-rotation to same Reference position, it is ensured that the match point between different images has similar angle, is divided according to the postrotational angle of characteristic point Area is polymerized, and can to the full extent keep the discrimination of image entirety characterization vector.
Accompanying drawing 1 is referred to, flow is realized for what the system embodiment was provided, its main flow is comprised the following steps：
Step 1：Using the camera acquisition target image for carrying gravity sensor, the image with gravitation information is obtained Sample, is stored in database；
Three reference axis of gravitation information including camera coordinate system respectively with the angle [g of gravity directionx(i),gy(i), gz(i)]；I is the i-th width image.
When initial, acceleration of gravity registration is [g during record mobile phone horizontal positionedx(0),gy(0),gz(0)], now gravity Direction is consistent with camera coordinates system Y-axis positive direction, remembers position on the basis of this position, and it is 0 to set this location drawing picture anglec of rotation.
Collection obtains database, and feature point extraction is carried out to all of image using common feature extraction algorithm.
Wherein, mobile phone shoots 640*480 image in different resolution as training sample database, and record shoots residing angle mobile phone every time Gravity sensor information and be stored in local document, major colleges and universities and commercial center adopt altogether in Beijing in the present embodiment At collection landmark 309, each building shoots 5, the picture of different angles, comprising different far and near scalings, different rotary angle with And different building sides are angularly.All images to shooting carry out feature extraction.
Step 2：Dextrorotation corner α during according to gravitation information computational intelligence terminal relative to horizontal positioned, will extract Image characteristic point angle rotated counterclockwise by angle α.
As shown in accompanying drawing 2 (b) camera coordinates system, gravity direction turns clockwise angle [alpha] in camera coordinates system, is revolved Image i ' after turning；
Because gravity is all the time perpendicular to ground, therefore there is rotated counterclockwise by angle α in real image, will should now extract All characteristic point angles increase α.
Step 3：The image characteristic point of all collections in image i ' is carried out into bag of words training, K cluster centre is obtained, That is to say visual dictionary.
All image characteristic points are clustered using the method for K mean cluster, and by K cluster centre storage in internal memory In.
Step 4：Feature coding is carried out to all characteristic points in image i ', characteristic point is mapped into visual dictionary, and each is clustered On center, and the characteristic point fallen on each cluster centre is carried out into subregion according to characteristic point angle, carried out on each subregion Characteristic aggregation, the overall compact vector for forming image is characterized.
B subregion is evenly dividing into according to characteristic point angle for each word on visual dictionary, each subregion is big It is small to beEach most like cluster centre is searched by KD Tree algorithms to all input picture characteristic points, if rotation The principal direction angle of the characteristic point after turning belongs to intervalJ=1 ..., B, then this feature point fall On j-th subregion.The last polymerization that local feature (characteristic point) is carried out on respective subregion.Characteristic vector after polymerization is figure The general token of picture.
Polymerization methodses use VLAD algorithms, that is, characteristic point to the residual error of current cluster centre is calculated, then to falling at each Residual error on subregion carries out cumulative summation：
Wherein xi,jIt is the characteristic point fallen on the subregion of j-th of ith cluster center, namely characteristic point is while meet nearest Neighbour is ith cluster center NN (x)=Ci, xi,jPrincipal direction anglePiFor it is poly- and to The subvector of amount.
Step 5：Repeat the above steps 1,2,4 to query image, the overall compact vector for forming query image is characterized, then Can be searched by arest neighbors and find the training image for most matching.
Wherein, when retrieval result test is done, all of image is selected simultaneously as training sample and query sample.Take Every preceding 5 arest neighbors result of test sample return that is to say that statistics Recall-5 is called together as the foundation for calculating retrieval result The rate of returning.Query image is rotated to reference position according to gravitation information carries out local feature polymerization, obtains the vector of query image Characterize.The COS distance of query image and training image is calculated, 5 maximums are most like with query image before remainder chordal distance Image.
Distance(x,yi)=cos (x, yi)
Recall rate Comparative result
Original VLAD Jegou This method
K=8 3.700971 3.697735 3.910032
K=16 3.855016 3.904854 4.042718
K=32 4.075728 4.091909 4.185113
K=64 4.255663 4.285437 4.337864
To sum up, presently preferred embodiments of the present invention is these are only, is not intended to limit the scope of the present invention.It is all Within the spirit and principles in the present invention, any modification, equivalent substitution and improvements made etc. should be included in protection of the invention Within the scope of.

Claims (3)

1. the characteristics of image querying method of gravity sensor and image characteristic point angle is combined, it is characterised in that including following step Suddenly：
Step 1：Using the camera acquisition target image for carrying gravity sensor, the image sample with gravitation information is obtained This, is stored in database；Three reference axis x-axis, y-axis, z of camera coordinate system when the gravitation information is specially shooting image Axle is respectively relative to the angle with gravity direction；Wherein z-axis is the axle perpendicular to video camera imaging face, the plane that x, y-axis are formed As video camera imaging face；
Y-axis positive direction is placed in it is consistent with gravity direction, will now camera position as reference position, at the reference position Gravitation information is [gx(0),gy(0),gz(0) it is], 0 with image rotation angle during reference position；
Step 2：Carry out the operation of following steps 201~204, shape as image i using each width image pattern in database successively Overall compact vector into all image patterns of correspondence is characterized；
Step 201, for image i, characteristic point is extracted to image pattern using feature extraction algorithm, image i is included in characteristic point Gravitation information be [gx(i),gy(i),gz(i)]；
Step 202, the gravity direction of image i turn clockwise angle [alpha] in camera coordinates systemiFor：
α i = g y ( i ) | g y ( i ) | * π 2 - a t a n g x ( i ) g y ( i ) , g y ( i ) ≠ 0 π 2 - g x ( i ) | g x ( i ) | * π 2 , g y ( i ) = 0
The angular nature value in all characteristic points in the image i that will be extracted all increases αi, obtain postrotational image i '；
Step 203, all of image characteristic point in image i ' is carried out bag of words training, obtain K cluster centre；
Step 204, feature coding is carried out to all of image characteristic point in image i ', characteristic point is mapped to each cluster centre On the region of the specified size at place, then the characteristic point fallen on each cluster centre is divided according to characteristic point angle Area, carries out characteristic aggregation on each subregion, and the overall compact vector for forming image pattern i ' is characterized；
Step 3：Using query image as image i, the operation of above-mentioned steps 201~204 is performed, form the whole of correspondence query image The compact vector of body is characterized；Method is searched using arest neighbors, is searched in image pattern in database and is most matched with the query image Image pattern.
2. the characteristics of image querying method of gravity sensor and image characteristic point angle is combined as claimed in claim 1, and it is special Levy and be, when using camera acquisition target image, from different far and near scalings, different rotary angle and different target side Angle be acquired.
3. the characteristics of image querying method of gravity sensor and image characteristic point angle is combined as claimed in claim 1, and it is special Levy and be, in the step 204, carry out using partial polymerization descriptor VLAD algorithms during characteristic aggregation on each subregion.
CN201510379232.7A 2015-07-01 2015-07-01 With reference to gravity sensor and the characteristics of image querying method of image characteristic point angle Active CN105045841B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510379232.7A CN105045841B (en) 2015-07-01 2015-07-01 With reference to gravity sensor and the characteristics of image querying method of image characteristic point angle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510379232.7A CN105045841B (en) 2015-07-01 2015-07-01 With reference to gravity sensor and the characteristics of image querying method of image characteristic point angle

Publications (2)

Publication Number Publication Date
CN105045841A CN105045841A (en) 2015-11-11
CN105045841B true CN105045841B (en) 2017-06-23

Family

ID=54452388

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510379232.7A Active CN105045841B (en) 2015-07-01 2015-07-01 With reference to gravity sensor and the characteristics of image querying method of image characteristic point angle

Country Status (1)

Country Link
CN (1) CN105045841B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106845492A (en) * 2016-10-17 2017-06-13 纳恩博（北京）科技有限公司 A kind of method and electronic equipment for extracting feature descriptor
CN108536769B (en) * 2018-03-22 2023-01-03 深圳市安软慧视科技有限公司 Image analysis method, search method and device, computer device and storage medium
CN110473229B (en) * 2019-08-21 2022-03-29 上海无线电设备研究所 Moving object detection method based on independent motion characteristic clustering
CN113495965A (en) * 2020-04-08 2021-10-12 百度在线网络技术（北京）有限公司 Multimedia content retrieval method, device, equipment and storage medium
CN114066781B (en) * 2022-01-18 2022-05-10 浙江鸿禾医疗科技有限责任公司 Capsule endoscope intestinal image identification and positioning method, storage medium and equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102254015A (en) * 2011-07-21 2011-11-23 上海交通大学 Image retrieval method based on visual phrases
CN102368237A (en) * 2010-10-18 2012-03-07 中国科学技术大学 Image retrieval method, device and system
CN104199842A (en) * 2014-08-07 2014-12-10 同济大学 Similar image retrieval method based on local feature neighborhood information
US8942515B1 (en) * 2012-10-26 2015-01-27 Lida Huang Method and apparatus for image retrieval

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102368237A (en) * 2010-10-18 2012-03-07 中国科学技术大学 Image retrieval method, device and system
CN102254015A (en) * 2011-07-21 2011-11-23 上海交通大学 Image retrieval method based on visual phrases
US8942515B1 (en) * 2012-10-26 2015-01-27 Lida Huang Method and apparatus for image retrieval
CN104199842A (en) * 2014-08-07 2014-12-10 同济大学 Similar image retrieval method based on local feature neighborhood information

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"On-Device Mobile Visual Location Recognition by Integrating Vision and Inertial Sensors";TaoGuan等;《IEEE Transactions on Multimedia》;20131130;第15卷(第7期);全文 *
"基于加权特征空间信息视觉词典的图像检索模型";董健;《计算机应用》;20140410(第4期);全文 *

Also Published As

Publication number Publication date
CN105045841A (en) 2015-11-11

Similar Documents

Publication Publication Date Title
CN105045841B (en) With reference to gravity sensor and the characteristics of image querying method of image characteristic point angle
CN106651942B (en) Three-dimensional rotating detection and rotary shaft localization method based on characteristic point
CN109887015B (en) Point cloud automatic registration method based on local curved surface feature histogram
CN106919944B (en) ORB algorithm-based large-view-angle image rapid identification method
CN110992263B (en) Image stitching method and system
CN110807781B (en) Point cloud simplifying method for retaining details and boundary characteristics
CN114972459B (en) Point cloud registration method based on low-dimensional point cloud local feature descriptor
CN106355577A (en) Method and system for quickly matching images on basis of feature states and global consistency
CN108388902B (en) Composite 3D descriptor construction method combining global framework point and local SHOT characteristics
CN111310821B (en) Multi-view feature fusion method, system, computer equipment and storage medium
CN108537844A (en) A kind of vision SLAM winding detection methods of fusion geological information
CN107590234B (en) RANSAC-based indoor visual positioning database redundant information reduction method
CN109086350B (en) Mixed image retrieval method based on WiFi
CN110197113B (en) Face detection method of high-precision anchor point matching strategy
CN109117851A (en) A kind of video image matching process based on lattice statistical constraint
CN115601574A (en) Unmanned aerial vehicle image matching method for improving AKAZE characteristics
CN114358166B (en) Multi-target positioning method based on self-adaptive k-means clustering
CN112614167A (en) Rock slice image alignment method combining single-polarization and orthogonal-polarization images
CN108320310A (en) Extraterrestrial target 3 d pose method of estimation based on image sequence
CN115512137A (en) Random stacked workpiece positioning method based on point cloud pose estimation
CN105139013A (en) Object recognition method integrating shape features and interest points
CN112418250B (en) Optimized matching method for complex 3D point cloud
CN102496022B (en) Effective feature point description I-BRIEF method
CN113283478B (en) Assembly body multi-view change detection method and device based on feature matching
CN109840525A (en) The extraction of circumference binary features with match searching method

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant