CN105653723B - A kind of query image feature method of cutting out for image retrieval - Google Patents
A kind of query image feature method of cutting out for image retrieval Download PDFInfo
- Publication number
- CN105653723B CN105653723B CN201610034219.2A CN201610034219A CN105653723B CN 105653723 B CN105653723 B CN 105653723B CN 201610034219 A CN201610034219 A CN 201610034219A CN 105653723 B CN105653723 B CN 105653723B
- Authority
- CN
- China
- Prior art keywords
- image
- visual word
- query image
- reference signal
- query
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000000007 visual effect Effects 0.000 claims abstract description 47
- 238000012795 verification Methods 0.000 claims description 4
- 230000006735 deficit Effects 0.000 claims description 2
- 238000000605 extraction Methods 0.000 claims description 2
- 238000013138 pruning Methods 0.000 abstract 1
- 238000003909 pattern recognition Methods 0.000 description 6
- 238000013139 quantization Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 2
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000013549 information retrieval technique Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000013179 statistical model Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Landscapes
- Engineering & Computer Science (AREA)
- Library & Information Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Processing Or Creating Images (AREA)
Abstract
本发明公开了一种用于图像检索的查询图像特征裁剪方法，其选择一个小的无关图像集的搜索结果作为参考信号。通过统计该信号中各个visual word对得分的贡献来对查询图像中的特征进行删减，这样能够在最后的得分中减去一些错误投票，对错误的结果进行压制。该方法能够对检索的性能起到促进的作用。本发明使用了对查询图像特征进行删减的方法，得到了更准确的搜索性能。使用无关图像的搜索结果作为参考信号，能够提升性能。
The invention discloses a query image feature cropping method for image retrieval, which selects a search result of a small irrelevant image set as a reference signal. The features in the query image are pruned by counting the contribution of each visual word in the signal to the score, so that some erroneous votes can be subtracted from the final score to suppress erroneous results. This method can promote the retrieval performance. The present invention uses the method of pruning the features of the query image to obtain more accurate search performance. Using the search results of irrelevant images as a reference signal can improve performance.
Description
Technical field
The present invention relates to computer vision, pattern-recognition, image retrieval, search engine field, and in particular to Yi Zhongyong
In the query image feature method of cutting out of image retrieval.
Background technique
With the development of internet, the data volume stored on network is skyrocketed through year by year, inspection of the Internet user to information
Rope demand is also growing.With the development of information retrieval technique and computer vision technique, allow user from internet
The image needed is obtained in real time.In such application, retrieving similar images technology occupies very big specific gravity.
Since the angle of image taking, environment have very big difference, and the information content in image is very big, therefore in real time
It is a very difficult thing that similar pictures are searched from mass picture.The present invention is in the unobvious feelings for improving time complexity
The performance that some image retrievals can be promoted under condition, has a very big significance.
The mainstream technology of image retrieval is based on SIFT feature (referring to document [1] D.G.Lowe, Distinctive at present
image features from scale-invariant keypoints,International Journal of
Computer Vision, vol.60, no.2, pp.91-110,2004.) and bag of words (Bag-of-words) to image into
Row expression, then completes retrieving using the technology of inverted index.
Such technology extracts some key areas from every sub-picture, and some high dimensional features are then extracted at key area,
Then quantified to obtain final expression using bag of words.It is calculated after the expression for completing all images using statistical model
The tf-idf weight of each visual word out, finally using inverted file retrieval (referring to document [2] J.Sivic,
A.Zisserman,Video Google:A text retrieval approach to object matching in
Videos, IEEE International Conference on Computer Vision, pp.1470-1477,2003. and
Document [3] J.Philbin, O.Chum, M.Isard, et al, Object retrieval with large
vocabularies and fast spatial matching,IEEE Conference on Computer Vision and
Pattern Recognition,pp.1-8,2007.)。
For some defects of this method, the method for some improving performances that supply a gap is suggested, and larger will not be promoted
Time consumption.And certain methods can be combined in compatible mode performance is made to obtain bigger promotion.
1) Philbin is (referring to document [3] J.Philbin, O.Chum, M.Isard, et al, Object retrieval
with large vocabularies and fast spatial matching,IEEE Conference on Computer
Vision and Pattern Recognition, pp.1-8,2007.) it proposed to complete on a large scale using dictionary in 2007
Image retrieval, search result is improved using the re-rank strategy of spatial information, this method is widely used as
Baseline is used.
2) Philbin is (referring to document [4] J.Philbin, O.Chum, M.Isard, et al, Lost in
quantization:Improving particular object retrieval in large scale image
Databases, pp.1-8,2008.) in 2008 propose Soft Assignment method solve the problems, such as quantization error.It will be every
A description quantization is to multiple Visual word.
3) Chum (referring to document [5] O.Chum, J.Philbin, J.Sivic, et al, Total recall:
Automatic query expansion with a generative feature model for object
Retrieval, IEEE International Conference on Computer Vision, pp.1-8,2007.) in
The method of Query expansion in 2007 promotes the recall rate of search, i.e., carries out information to query using initial search result
Supplement.
4) J é gou is (referring to document [6] H.J é gou, M.Douze, C.Schmid, Improving bag-of-features
for large scale image search,International Journal of Computer Vision,vol.87,
No.3, pp.316-336,2010.) the Hamming Embedding method that reduction quantization error influences was proposed in 2008,
Each SIFT description is also bound with a binary coding, the binary coding other than quantifying to a Visual word
It is related with the residual error of Visual word to description.
5) J é gou is (referring to document [7] H.J é gou, M.Douze, C.Schmid, On the burstiness of
visual elements,IEEE Conference on Computer Vision and Pattern Recognition,
Pp.1169-1176,2009.) proposed in the picture, often there are many word maximum probability can repeatedly occur very much in 2009.Jé
Gou is counted on the basis of Hamming Embedding using more detailed quantitative information, and Intra- is solved
Burstiness (in image) and Inter-burstiness (between image) problem.
6) Chum is (referring to document [8] O.Chum, J.Matas.Unsupervised discovery of co-
occurrence in sparse high dimensional data,IEEE Conference on Computer Vision
And Pattern Recognition, pp.3416-3423,2010.) it proposed to pass through the total hair for detecting some word in 2010
Property is come come the repetition score that reduces some repeat elements.
7) Zheng is (referring to document [9] L.Zheng, S.Wang, Z.Liu, et al, Lp-norm idf for large
scale image search,IEEE Conference on Computer Vision and Pattern
Recognition, pp.1626-1633,2013.) it proposed to consider tf weight when calculating idf weight in 2013, it is each to calculate
The weight of a word.
Select the search result of a small unrelated images collection as reference signal in the present invention.By counting in the signal
Each visual word deletes the feature in query image the contribution of score, in this way can be in last score
In subtract it is some mistake ballot, to mistake result suppress.This method can play the role of promotion to the performance of retrieval.
Summary of the invention
It is an object of the invention to: 1) it is able to use the performance for the method promotion picture search that feature is deleted；2) can not
Significantly improve time complexity；3) memory dosage can not be significantly improved；4) method is not excessively complicated, can keep with other methods
Good compatibility.
The technical solution adopted by the present invention are as follows: a kind of query image feature method of cutting out for image retrieval, this method
Include the following steps:
Step 1: extracting SIFT feature to query image and being quantified using bag of words, then in big image to be retrieved
Search and query image similar image under COS distance in library, and to its ranking；
Step 2: the contribution number of each visual word in image score wherein in the top is counted, as positive letter
Number；
Step 3: search and query image similar image under COS distance in unrelated images library, and to its ranking；
Step 4: the contribution number of each visual word in image score wherein in the top is counted, as negative letter
Number；
Step 5: calculating the association relationship of each visual word in negative signal according to the statistical result of positive negative signal, press
Value size sequence, the biggish some visual word of value；
Step 6: subtracting these features in query image, and using the feature after deleting in big image library to be retrieved
Middle search obtains final result.
Wherein, the search in a kind of image library to be retrieved of step is specific as follows:
To the region Hessian-affine is extracted in query image, then in the SIFT of each extracted region one 128 dimension
Then feature is quantified using dictionary trained in bag of words, obtain the final expression of image, this is expressed as
The set of the ID of visual word；Then it is scanned in database to be retrieved using the image expression, obtains search knot
Fruit, by Spatial verification, the search result for taking its in the top is as positive reference signal.
Wherein, the contribution number of each visual word in positive reference signal is counted in step 2 specifically:
Image in the top in real library to be retrieved is obtained in step 1, step 2 counts the score of these images
The more some visual word of middle contribution frequency, these images are by spatial verification, it is assumed that they
It is correct search result, the mode of these visual word is to be worth retaining.
Wherein, the search in step 3 in unrelated images library specifically:
It is scanned in some natural image libraries randomly selected in a network, all images and query image in the network
Unrelated, the search result for taking its in the top is as negative reference signal.
Wherein, step 4 counts the contribution number of each visual word in negative reference signal specifically:
Count the more some visual word of contribution frequency in negative reference signal, the mode of these visual word
It may be considered that pair the result is that there is impairment.
Wherein, mutual information is calculated in step 5 specifically:
The mutual information of the visual word occurred in negative reference signal is calculated in conjunction with the statistical result of positive and negative reference signal
Value, takes the biggish some visual word of its value.
Wherein, step 6 kind completes retrieval using the feature after deleting specifically:
The visual word obtained on last stage is subtracted in query image, then using the feature after deleting real
Retrieval is completed in image library to be retrieved, exports search result.
The present invention compared with prior art the advantages of and good effect are as follows:
1, the method deleted query image feature has been used, more accurate search performance has been obtained.
2, use the search result of unrelated images as reference signal, be able to ascend performance.
3, smaller to the promotion of the time complexity of algorithm and space complexity.
4, better performance can be obtained than single negative reference signal using government's reference signal in algorithm.
5, feature is deleted using mutual information as standard, simply, speed is fast and effect is good.
Detailed description of the invention
Fig. 1 is a kind of query image feature method of cutting out flow chart for image retrieval of the present invention.
Specific embodiment
With reference to the accompanying drawing and specific embodiment further illustrates the present invention.
By the present invention in that use search result in unrelated images library as negative reference signal, to the feature of query image into
Row is deleted, and last search is then carried out using the feature after deleting.
1. the initial retrieval in database to be retrieved
The first time search operation in image library to be retrieved is completed using traditional method in the present invention.To query image
The region Hessian-affine is extracted, then to the SIFT feature of each extracted region 128 dimension, then using trained in advance
Dictionary in bag of words quantifies each SIFT feature, obtains image expression to the end.Here quantization uses
Arest neighbors lookup algorithm in FLANN open source library.Also traditional inverted index method is used when search, while also using tf-idf
Weight.Image search result in the top is counted as positive reference signal.
2. counting the contribution number of each visual word in positive reference signal
Still need to count the number that each visual word occurs in positive reference signal, each visual word in the present invention
Primary to the search score contribution of arbitrary image, frequency of occurrence adds 1.
3. the retrieval in unrelated images library
It also needs to be retrieved in the unrelated images library collected in advance in inventive algorithm, feature extraction and search strategy
As hereinbefore, image search result in the top is counted as negative reference signal.Information in the signal is considered as nothing
With and search result can be caused harm.
4. counting the contribution number of each visual word in negative reference signal
The number occurred with visual word each in the aforementioned same negative reference signal of method statistic.
5. the calculating of mutual information
The calculation formula of mutual information is as follows:
I(x:i, y) and=H (y)+H (x:i)-H(x:i,y)
Wherein x:iFor the feature of i-th dimension, each visual word, y are corresponded to here for the label of signal, in positive signal
Be 1, in negative signal for 0, H be entropy function.
Formula can calculate the association relationship of each visual word in negative reference signal according to this, select its intermediate value larger
Some visual word be used as feature to be deleted.
6. completing retrieval using the feature after deleting
It is retrieved again in database to be retrieved using the query image feature after deleting, this time searching for obtained result is
Final search result.
Algorithm overall flow figure such as Fig. 1.
1) image to be checked is read.
2) to the region image zooming-out Hessian-affine, in each extracted region SIFT feature, bag of words mould is then used
Type quantization.
3) it is scanned in image library to be retrieved, is used herein as inverted index.
4) image in the top is counted, as positive reference signal.
5) the contribution number of each visual word in positive reference signal is counted.
6) it is scanned in unrelated images library, is used herein as inverted index.
7) image in the top is counted, as negative reference signal.
8) the contribution number of each visual word in negative reference signal is counted.
9) according to positive negative signal, the mutual information of each visual word in negative signal is calculated.
10) the biggish some visual word of mutual information intermediate value are counted, it is removed from query feature.
11) it is scanned in image library to be retrieved using the query feature after deleting.
12) search result is exported.
Claims (7)
1. a kind of query image feature method of cutting out for image retrieval, it is characterised in that: this method comprises the following steps:
Step 1: extracting SIFT feature to query image and being quantified using bag of words, then in big image library to be retrieved
Search and query image similar image under COS distance, and to its ranking；
Step 2: the contribution number of each visual word in image score wherein in the top is counted, as positive signal；
Step 3: search and query image similar image under COS distance in unrelated images library, and to its ranking；
Step 4: the contribution number of each visual word in image score wherein in the top is counted, as negative signal；
Step 5: the association relationship of each visual word in negative signal is calculated according to the statistical result of positive negative signal, by the value
Size sequence, the biggish some visual word of value；
Step 6: subtracting these features in query image, and searched in big image library to be retrieved using the feature after deleting
Rope obtains final result.
2. a kind of query image feature method of cutting out for image retrieval according to claim 1, it is characterised in that: step
Search in rapid one in image library to be retrieved is specific as follows:
To the extraction region Hessian-affine in query image, the SIFT feature then tieed up at each extracted region one 128,
Then quantified using dictionary trained in bag of words, obtain the final expression of image, this is expressed as some visual
The set of the ID of word；Then it is scanned in database to be retrieved using the image expression, obtains search result, passed through
Spatial verification, the search result for taking its in the top is as positive reference signal.
3. a kind of query image feature method of cutting out for image retrieval according to claim 1, it is characterised in that: step
The contribution number of each visual word in positive reference signal is counted in rapid two specifically:
Image in the top in real library to be retrieved is obtained in step 1, step 2 counts tribute in the score of these images
The more some visual word of frequency are offered, these images are by spatial verification, if they are correct
Search result, then retain the mode of these visual word.
4. a kind of query image feature method of cutting out for image retrieval according to claim 1, it is characterised in that: step
Search in rapid three in unrelated images library specifically:
Scanned in some natural image libraries randomly selected in a network, in the network all images and query image without
It closes, the search result for taking its in the top is as negative reference signal.
5. a kind of query image feature method of cutting out for image retrieval according to claim 1, it is characterised in that: step
The contribution number of each visual word in the negative reference signal of rapid four statistics specifically:
The more some visual word of contribution frequency in negative reference signal are counted, the mode of these visual word is considered
Pair the result is that there is impairment.
6. a kind of query image feature method of cutting out for image retrieval according to claim 1, it is characterised in that: step
Mutual information is calculated in rapid five specifically:
The association relationship that the visual word occurred in negative reference signal is calculated in conjunction with the statistical result of positive and negative reference signal, takes
It is worth biggish some visual word.
7. a kind of query image feature method of cutting out for image retrieval according to claim 1, it is characterised in that: step
Retrieval is completed using the feature after deleting in rapid six specifically:
The visual word obtained on last stage is subtracted in query image, then using the feature after deleting really to be checked
Retrieval is completed in the image library of rope, exports search result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610034219.2A CN105653723B (en) | 2016-01-19 | 2016-01-19 | A kind of query image feature method of cutting out for image retrieval |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610034219.2A CN105653723B (en) | 2016-01-19 | 2016-01-19 | A kind of query image feature method of cutting out for image retrieval |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105653723A CN105653723A (en) | 2016-06-08 |
| CN105653723B true CN105653723B (en) | 2019-03-01 |
Family
ID=56486832
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610034219.2A Active CN105653723B (en) | 2016-01-19 | 2016-01-19 | A kind of query image feature method of cutting out for image retrieval |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105653723B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102542014A (en) * | 2011-12-16 | 2012-07-04 | 华中科技大学 | Image searching feedback method based on contents |
| CN103440262A (en) * | 2013-07-31 | 2013-12-11 | 东莞中山大学研究院 | Image Retrieval System and Method Based on Correlation Feedback and Bag-of-Features |
| CN104077344A (en) * | 2013-12-31 | 2014-10-01 | 河南大学 | Self-adaption learning region importance based interactive image retrieval method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9405773B2 (en) * | 2010-03-29 | 2016-08-02 | Ebay Inc. | Searching for more products like a specified product |
-
2016
- 2016-01-19 CN CN201610034219.2A patent/CN105653723B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102542014A (en) * | 2011-12-16 | 2012-07-04 | 华中科技大学 | Image searching feedback method based on contents |
| CN103440262A (en) * | 2013-07-31 | 2013-12-11 | 东莞中山大学研究院 | Image Retrieval System and Method Based on Correlation Feedback and Bag-of-Features |
| CN104077344A (en) * | 2013-12-31 | 2014-10-01 | 河南大学 | Self-adaption learning region importance based interactive image retrieval method and system |
Non-Patent Citations (1)
| Title |
|---|
| "图像检索中的图像表达方法研究";杜彬彬;《中国优秀硕士学位论文全文数据库 信息科技辑》;20170110;第I138-516页 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105653723A (en) | 2016-06-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zheng et al. | SIFT meets CNN: A decade survey of instance retrieval | |
| Jing et al. | Pagerank for product image search | |
| Li et al. | Object bank: An object-level image representation for high-level visual recognition | |
| Chum et al. | Total recall II: Query expansion revisited | |
| Clinchant et al. | Semantic combination of textual and visual information in multimedia retrieval | |
| Bruni et al. | Distributional semantics from text and images | |
| US9087297B1 (en) | Accurate video concept recognition via classifier combination | |
| CN102053991B (en) | Method and system for multi-language document retrieval | |
| US20110116690A1 (en) | Automatically Mining Person Models of Celebrities for Visual Search Applications | |
| CN102549603A (en) | Relevance-based image selection | |
| CN102254015A (en) | Image retrieval method based on visual phrases | |
| CN108268539A (en) | Video matching system based on text analyzing | |
| CN103425640A (en) | Multimedia questioning-answering system and method | |
| Le et al. | NII-HITACHI-UIT at TRECVID 2016. | |
| CN105849720A (en) | Visual semantic complex network and method for forming network | |
| CN105989001B (en) | Image search method and device, image search system | |
| CN104537028B (en) | A kind of Web information processing method and device | |
| CN109446399A (en) | A kind of video display entity search method | |
| CN111986259A (en) | Training method of character and face detection model, auditing method of video data and related device | |
| JP6017277B2 (en) | Program, apparatus and method for calculating similarity between contents represented by set of feature vectors | |
| Mazloom et al. | Few-example video event retrieval using tag propagation | |
| CN105653723B (en) | A kind of query image feature method of cutting out for image retrieval | |
| Imran et al. | Event recognition from photo collections via pagerank | |
| CN114912026B (en) | A kind of network public opinion monitoring, analysis and processing method, equipment and computer storage medium | |
| Zhou et al. | A practical spatial re-ranking method for instance search from videos |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220608 Address after: 230093 room 1701, block C, building 1, zone J, phase II, Hefei Innovation Industrial Park, No. 2800, innovation Avenue, high tech Zone, Hefei, Anhui Patentee after: SNEGRID ELECTRIC TECHNOLOGY Co.,Ltd. Address before: 230026 Jinzhai Road, Baohe District, Hefei, Anhui Province, No. 96 Patentee before: University of Science and Technology of China |
|
| TR01 | Transfer of patent right |