CN105653723B - A kind of query image feature method of cutting out for image retrieval - Google Patents

A kind of query image feature method of cutting out for image retrieval Download PDF

Info

Publication number
CN105653723B
CN105653723B CN201610034219.2A CN201610034219A CN105653723B CN 105653723 B CN105653723 B CN 105653723B CN 201610034219 A CN201610034219 A CN 201610034219A CN 105653723 B CN105653723 B CN 105653723B
Authority
CN
China
Prior art keywords
image
visual word
query image
reference signal
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201610034219.2A
Other languages
Chinese (zh)
Other versions
CN105653723A (en
Inventor
凌强
杜彬彬
李峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Snegrid Electric Technology Co ltd
Original Assignee
University of Science and Technology of China USTC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology of China USTC filed Critical University of Science and Technology of China USTC
Priority to CN201610034219.2A priority Critical patent/CN105653723B/en
Publication of CN105653723A publication Critical patent/CN105653723A/en
Application granted granted Critical
Publication of CN105653723B publication Critical patent/CN105653723B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content

Landscapes

  • Engineering & Computer Science (AREA)
  • Library & Information Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention discloses a kind of query image feature method of cutting out for image retrieval, select the search result of a small unrelated images collection as reference signal.The feature in query image is deleted by counting each visual word in the signal to the contribution of score, some mistake ballots can be subtracted in last score in this way, the result of mistake is suppressed.This method can play the role of promotion to the performance of retrieval.Present invention uses the methods deleted query image feature, have obtained more accurate search performance.Use the search result of unrelated images as reference signal, is able to ascend performance.

Description

A kind of query image feature method of cutting out for image retrieval
Technical field
The present invention relates to computer vision, pattern-recognition, image retrieval, search engine field, and in particular to Yi Zhongyong In the query image feature method of cutting out of image retrieval.
Background technique
With the development of internet, the data volume stored on network is skyrocketed through year by year, inspection of the Internet user to information Rope demand is also growing.With the development of information retrieval technique and computer vision technique, allow user from internet The image needed is obtained in real time.In such application, retrieving similar images technology occupies very big specific gravity.
Since the angle of image taking, environment have very big difference, and the information content in image is very big, therefore in real time It is a very difficult thing that similar pictures are searched from mass picture.The present invention is in the unobvious feelings for improving time complexity The performance that some image retrievals can be promoted under condition, has a very big significance.
The mainstream technology of image retrieval is based on SIFT feature (referring to document [1] D.G.Lowe, Distinctive at present image features from scale-invariant keypoints,International Journal of Computer Vision, vol.60, no.2, pp.91-110,2004.) and bag of words (Bag-of-words) to image into Row expression, then completes retrieving using the technology of inverted index.
Such technology extracts some key areas from every sub-picture, and some high dimensional features are then extracted at key area, Then quantified to obtain final expression using bag of words.It is calculated after the expression for completing all images using statistical model The tf-idf weight of each visual word out, finally using inverted file retrieval (referring to document [2] J.Sivic, A.Zisserman,Video Google:A text retrieval approach to object matching in Videos, IEEE International Conference on Computer Vision, pp.1470-1477,2003. and Document [3] J.Philbin, O.Chum, M.Isard, et al, Object retrieval with large vocabularies and fast spatial matching,IEEE Conference on Computer Vision and Pattern Recognition,pp.1-8,2007.)。
For some defects of this method, the method for some improving performances that supply a gap is suggested, and larger will not be promoted Time consumption.And certain methods can be combined in compatible mode performance is made to obtain bigger promotion.
1) Philbin is (referring to document [3] J.Philbin, O.Chum, M.Isard, et al, Object retrieval with large vocabularies and fast spatial matching,IEEE Conference on Computer Vision and Pattern Recognition, pp.1-8,2007.) it proposed to complete on a large scale using dictionary in 2007 Image retrieval, search result is improved using the re-rank strategy of spatial information, this method is widely used as Baseline is used.
2) Philbin is (referring to document [4] J.Philbin, O.Chum, M.Isard, et al, Lost in quantization:Improving particular object retrieval in large scale image Databases, pp.1-8,2008.) in 2008 propose Soft Assignment method solve the problems, such as quantization error.It will be every A description quantization is to multiple Visual word.
3) Chum (referring to document [5] O.Chum, J.Philbin, J.Sivic, et al, Total recall: Automatic query expansion with a generative feature model for object Retrieval, IEEE International Conference on Computer Vision, pp.1-8,2007.) in The method of Query expansion in 2007 promotes the recall rate of search, i.e., carries out information to query using initial search result Supplement.
4) J é gou is (referring to document [6] H.J é gou, M.Douze, C.Schmid, Improving bag-of-features for large scale image search,International Journal of Computer Vision,vol.87, No.3, pp.316-336,2010.) the Hamming Embedding method that reduction quantization error influences was proposed in 2008, Each SIFT description is also bound with a binary coding, the binary coding other than quantifying to a Visual word It is related with the residual error of Visual word to description.
5) J é gou is (referring to document [7] H.J é gou, M.Douze, C.Schmid, On the burstiness of visual elements,IEEE Conference on Computer Vision and Pattern Recognition, Pp.1169-1176,2009.) proposed in the picture, often there are many word maximum probability can repeatedly occur very much in 2009.Jé Gou is counted on the basis of Hamming Embedding using more detailed quantitative information, and Intra- is solved Burstiness (in image) and Inter-burstiness (between image) problem.
6) Chum is (referring to document [8] O.Chum, J.Matas.Unsupervised discovery of co- occurrence in sparse high dimensional data,IEEE Conference on Computer Vision And Pattern Recognition, pp.3416-3423,2010.) it proposed to pass through the total hair for detecting some word in 2010 Property is come come the repetition score that reduces some repeat elements.
7) Zheng is (referring to document [9] L.Zheng, S.Wang, Z.Liu, et al, Lp-norm idf for large scale image search,IEEE Conference on Computer Vision and Pattern Recognition, pp.1626-1633,2013.) it proposed to consider tf weight when calculating idf weight in 2013, it is each to calculate The weight of a word.
Select the search result of a small unrelated images collection as reference signal in the present invention.By counting in the signal Each visual word deletes the feature in query image the contribution of score, in this way can be in last score In subtract it is some mistake ballot, to mistake result suppress.This method can play the role of promotion to the performance of retrieval.
Summary of the invention
It is an object of the invention to: 1) it is able to use the performance for the method promotion picture search that feature is deleted；2) can not Significantly improve time complexity；3) memory dosage can not be significantly improved；4) method is not excessively complicated, can keep with other methods Good compatibility.
The technical solution adopted by the present invention are as follows: a kind of query image feature method of cutting out for image retrieval, this method Include the following steps:
Step 1: extracting SIFT feature to query image and being quantified using bag of words, then in big image to be retrieved Search and query image similar image under COS distance in library, and to its ranking；
Step 2: the contribution number of each visual word in image score wherein in the top is counted, as positive letter Number；
Step 3: search and query image similar image under COS distance in unrelated images library, and to its ranking；
Step 4: the contribution number of each visual word in image score wherein in the top is counted, as negative letter Number；
Step 5: calculating the association relationship of each visual word in negative signal according to the statistical result of positive negative signal, press Value size sequence, the biggish some visual word of value；
Step 6: subtracting these features in query image, and using the feature after deleting in big image library to be retrieved Middle search obtains final result.
Wherein, the search in a kind of image library to be retrieved of step is specific as follows:
To the region Hessian-affine is extracted in query image, then in the SIFT of each extracted region one 128 dimension Then feature is quantified using dictionary trained in bag of words, obtain the final expression of image, this is expressed as The set of the ID of visual word；Then it is scanned in database to be retrieved using the image expression, obtains search knot Fruit, by Spatial verification, the search result for taking its in the top is as positive reference signal.
Wherein, the contribution number of each visual word in positive reference signal is counted in step 2 specifically:
Image in the top in real library to be retrieved is obtained in step 1, step 2 counts the score of these images The more some visual word of middle contribution frequency, these images are by spatial verification, it is assumed that they It is correct search result, the mode of these visual word is to be worth retaining.
Wherein, the search in step 3 in unrelated images library specifically:
It is scanned in some natural image libraries randomly selected in a network, all images and query image in the network Unrelated, the search result for taking its in the top is as negative reference signal.
Wherein, step 4 counts the contribution number of each visual word in negative reference signal specifically:
Count the more some visual word of contribution frequency in negative reference signal, the mode of these visual word It may be considered that pair the result is that there is impairment.
Wherein, mutual information is calculated in step 5 specifically:
The mutual information of the visual word occurred in negative reference signal is calculated in conjunction with the statistical result of positive and negative reference signal Value, takes the biggish some visual word of its value.
Wherein, step 6 kind completes retrieval using the feature after deleting specifically:
The visual word obtained on last stage is subtracted in query image, then using the feature after deleting real Retrieval is completed in image library to be retrieved, exports search result.
The present invention compared with prior art the advantages of and good effect are as follows:
1, the method deleted query image feature has been used, more accurate search performance has been obtained.
2, use the search result of unrelated images as reference signal, be able to ascend performance.
3, smaller to the promotion of the time complexity of algorithm and space complexity.
4, better performance can be obtained than single negative reference signal using government's reference signal in algorithm.
5, feature is deleted using mutual information as standard, simply, speed is fast and effect is good.
Detailed description of the invention
Fig. 1 is a kind of query image feature method of cutting out flow chart for image retrieval of the present invention.
Specific embodiment
With reference to the accompanying drawing and specific embodiment further illustrates the present invention.
By the present invention in that use search result in unrelated images library as negative reference signal, to the feature of query image into Row is deleted, and last search is then carried out using the feature after deleting.
1. the initial retrieval in database to be retrieved
The first time search operation in image library to be retrieved is completed using traditional method in the present invention.To query image The region Hessian-affine is extracted, then to the SIFT feature of each extracted region 128 dimension, then using trained in advance Dictionary in bag of words quantifies each SIFT feature, obtains image expression to the end.Here quantization uses Arest neighbors lookup algorithm in FLANN open source library.Also traditional inverted index method is used when search, while also using tf-idf Weight.Image search result in the top is counted as positive reference signal.
2. counting the contribution number of each visual word in positive reference signal
Still need to count the number that each visual word occurs in positive reference signal, each visual word in the present invention Primary to the search score contribution of arbitrary image, frequency of occurrence adds 1.
3. the retrieval in unrelated images library
It also needs to be retrieved in the unrelated images library collected in advance in inventive algorithm, feature extraction and search strategy As hereinbefore, image search result in the top is counted as negative reference signal.Information in the signal is considered as nothing With and search result can be caused harm.
4. counting the contribution number of each visual word in negative reference signal
The number occurred with visual word each in the aforementioned same negative reference signal of method statistic.
5. the calculating of mutual information
The calculation formula of mutual information is as follows:
I(x:i, y) and=H (y)+H (x:i)-H(x:i,y)
Wherein x:iFor the feature of i-th dimension, each visual word, y are corresponded to here for the label of signal, in positive signal Be 1, in negative signal for 0, H be entropy function.
Formula can calculate the association relationship of each visual word in negative reference signal according to this, select its intermediate value larger Some visual word be used as feature to be deleted.
6. completing retrieval using the feature after deleting
It is retrieved again in database to be retrieved using the query image feature after deleting, this time searching for obtained result is Final search result.
Algorithm overall flow figure such as Fig. 1.
1) image to be checked is read.
2) to the region image zooming-out Hessian-affine, in each extracted region SIFT feature, bag of words mould is then used Type quantization.
3) it is scanned in image library to be retrieved, is used herein as inverted index.
4) image in the top is counted, as positive reference signal.
5) the contribution number of each visual word in positive reference signal is counted.
6) it is scanned in unrelated images library, is used herein as inverted index.
7) image in the top is counted, as negative reference signal.
8) the contribution number of each visual word in negative reference signal is counted.
9) according to positive negative signal, the mutual information of each visual word in negative signal is calculated.
10) the biggish some visual word of mutual information intermediate value are counted, it is removed from query feature.
11) it is scanned in image library to be retrieved using the query feature after deleting.
12) search result is exported.

Claims (7)

1. a kind of query image feature method of cutting out for image retrieval, it is characterised in that: this method comprises the following steps:
Step 1: extracting SIFT feature to query image and being quantified using bag of words, then in big image library to be retrieved Search and query image similar image under COS distance, and to its ranking；
Step 2: the contribution number of each visual word in image score wherein in the top is counted, as positive signal；
Step 3: search and query image similar image under COS distance in unrelated images library, and to its ranking；
Step 4: the contribution number of each visual word in image score wherein in the top is counted, as negative signal；
Step 5: the association relationship of each visual word in negative signal is calculated according to the statistical result of positive negative signal, by the value Size sequence, the biggish some visual word of value；
Step 6: subtracting these features in query image, and searched in big image library to be retrieved using the feature after deleting Rope obtains final result.
2. a kind of query image feature method of cutting out for image retrieval according to claim 1, it is characterised in that: step Search in rapid one in image library to be retrieved is specific as follows:
To the extraction region Hessian-affine in query image, the SIFT feature then tieed up at each extracted region one 128, Then quantified using dictionary trained in bag of words, obtain the final expression of image, this is expressed as some visual The set of the ID of word；Then it is scanned in database to be retrieved using the image expression, obtains search result, passed through Spatial verification, the search result for taking its in the top is as positive reference signal.
3. a kind of query image feature method of cutting out for image retrieval according to claim 1, it is characterised in that: step The contribution number of each visual word in positive reference signal is counted in rapid two specifically:
Image in the top in real library to be retrieved is obtained in step 1, step 2 counts tribute in the score of these images The more some visual word of frequency are offered, these images are by spatial verification, if they are correct Search result, then retain the mode of these visual word.
4. a kind of query image feature method of cutting out for image retrieval according to claim 1, it is characterised in that: step Search in rapid three in unrelated images library specifically:
Scanned in some natural image libraries randomly selected in a network, in the network all images and query image without It closes, the search result for taking its in the top is as negative reference signal.
5. a kind of query image feature method of cutting out for image retrieval according to claim 1, it is characterised in that: step The contribution number of each visual word in the negative reference signal of rapid four statistics specifically:
The more some visual word of contribution frequency in negative reference signal are counted, the mode of these visual word is considered Pair the result is that there is impairment.
6. a kind of query image feature method of cutting out for image retrieval according to claim 1, it is characterised in that: step Mutual information is calculated in rapid five specifically:
The association relationship that the visual word occurred in negative reference signal is calculated in conjunction with the statistical result of positive and negative reference signal, takes It is worth biggish some visual word.
7. a kind of query image feature method of cutting out for image retrieval according to claim 1, it is characterised in that: step Retrieval is completed using the feature after deleting in rapid six specifically:
The visual word obtained on last stage is subtracted in query image, then using the feature after deleting really to be checked Retrieval is completed in the image library of rope, exports search result.
CN201610034219.2A 2016-01-19 2016-01-19 A kind of query image feature method of cutting out for image retrieval Active CN105653723B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610034219.2A CN105653723B (en) 2016-01-19 2016-01-19 A kind of query image feature method of cutting out for image retrieval

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610034219.2A CN105653723B (en) 2016-01-19 2016-01-19 A kind of query image feature method of cutting out for image retrieval

Publications (2)

Publication Number Publication Date
CN105653723A CN105653723A (en) 2016-06-08
CN105653723B true CN105653723B (en) 2019-03-01

Family

ID=56486832

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610034219.2A Active CN105653723B (en) 2016-01-19 2016-01-19 A kind of query image feature method of cutting out for image retrieval

Country Status (1)

Country Link
CN (1) CN105653723B (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102542014A (en) * 2011-12-16 2012-07-04 华中科技大学 Image searching feedback method based on contents
CN103440262A (en) * 2013-07-31 2013-12-11 东莞中山大学研究院 Image searching system and image searching method basing on relevance feedback and Bag-of-Features
CN104077344A (en) * 2013-12-31 2014-10-01 河南大学 Self-adaption learning region importance based interactive image retrieval method and system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9405773B2 (en) * 2010-03-29 2016-08-02 Ebay Inc. Searching for more products like a specified product

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102542014A (en) * 2011-12-16 2012-07-04 华中科技大学 Image searching feedback method based on contents
CN103440262A (en) * 2013-07-31 2013-12-11 东莞中山大学研究院 Image searching system and image searching method basing on relevance feedback and Bag-of-Features
CN104077344A (en) * 2013-12-31 2014-10-01 河南大学 Self-adaption learning region importance based interactive image retrieval method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
"图像检索中的图像表达方法研究";杜彬彬;《中国优秀硕士学位论文全文数据库 信息科技辑》;20170110;第I138-516页

Also Published As

Publication number Publication date
CN105653723A (en) 2016-06-08

Similar Documents

Publication Publication Date Title
Zheng et al. SIFT meets CNN: A decade survey of instance retrieval
WO2022068196A1 (en) Cross-modal data processing method and device, storage medium, and electronic device
US11146862B2 (en) Generating tags for a digital video
Jing et al. Pagerank for product image search
Li et al. Object bank: An object-level image representation for high-level visual recognition
Chum et al. Total recall II: Query expansion revisited
CN104050247B (en) The method for realizing massive video quick-searching
Wang et al. Contextual weighting for vocabulary tree based image retrieval
Clinchant et al. Semantic combination of textual and visual information in multimedia retrieval
Bruni et al. Distributional semantics from text and images
US9087297B1 (en) Accurate video concept recognition via classifier combination
CN102053991B (en) Method and system for multi-language document retrieval
US20110116690A1 (en) Automatically Mining Person Models of Celebrities for Visual Search Applications
Xie et al. Fast and accurate near-duplicate image search with affinity propagation on the ImageWeb
Ueki et al. Waseda_Meisei at TRECVID 2017: Ad-hoc Video Search.
WO2013170587A1 (en) Multimedia question and answer system and method
Weyand et al. Visual landmark recognition from internet photo collections: A large-scale evaluation
Le et al. NII-HITACHI-UIT at TRECVID 2016.
CN105849720A (en) Visual semantic complex network and method for forming network
CN110502664A (en) Video tab indexes base establishing method, video tab generation method and device
CN109446399A (en) A kind of video display entity search method
CN105760875A (en) Binary image feature similarity discrimination method based on random forest algorithm
Mazloom et al. Few-example video event retrieval using tag propagation
JP6017277B2 (en) Program, apparatus and method for calculating similarity between contents represented by set of feature vectors
Imran et al. Event recognition from photo collections via pagerank

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20220608

Address after: 230093 room 1701, block C, building 1, zone J, phase II, Hefei Innovation Industrial Park, No. 2800, innovation Avenue, high tech Zone, Hefei, Anhui

Patentee after: SNEGRID ELECTRIC TECHNOLOGY Co.,Ltd.

Address before: 230026 Jinzhai Road, Baohe District, Hefei, Anhui Province, No. 96

Patentee before: University of Science and Technology of China