CN105989128B - A kind of method and device of image retrieval - Google Patents
A kind of method and device of image retrieval Download PDFInfo
- Publication number
- CN105989128B CN105989128B CN201510083238.XA CN201510083238A CN105989128B CN 105989128 B CN105989128 B CN 105989128B CN 201510083238 A CN201510083238 A CN 201510083238A CN 105989128 B CN105989128 B CN 105989128B
- Authority
- CN
- China
- Prior art keywords
- description
- samples pictures
- input picture
- similarity
- vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Analysis (AREA)
Abstract
The present invention is suitable for information technology field, provides a kind of method and device of image retrieval, comprising: obtains first description of every samples pictures and the second description of input picture；Sub and described second description of first description is converted into vector form only comprising " 0 " and " 1 " by Fisher vector form respectively；The similarity of the second description and the first description after each conversion after calculating separately conversion, screens the samples pictures according to calculated result, obtains the search result of the input picture.Description of picture is converted to vector only comprising element " 0 " and element " 1 " by Fisher vector by the present invention, to carry out preliminary screening to picture before carrying out inner product of vectors operation, reduce the samples pictures quantity for carrying out inner product of vectors operation, to greatly improve the arithmetic speed of image retrieval in large-scale picture retrieval system, recall precision is improved.
Description
Technical field
The invention belongs to information technology field more particularly to a kind of method and devices of image retrieval.
Background technique
With the rapid development of digital technology, sensing technology and network technology, the quantity of picture is more and more, and content is also got over
Come abundanter, content-based image retrieval mode starts to grow up, by initial raw based on colouring information and shape information
At description to be retrieved, develops to and retrieved based on BOW (bag of words) model, by feature point extraction, and
Histogram is generated after clustering to the characteristic point of extraction, using description as every width picture, for carrying out similarity mode,
Complete image retrieval.
Based on BOW model, the prior art is generated using gauss hybrid models (Gaussian Mixture Model, GMM)
Description of Fisher vector form, however, in order to not influence the precision of retrieval, being generated with the increase of picture number
The number of the Gauss model to be used of Fisher vector is also required to increase, and leading to the dimension of last Fisher vector is linearly to increase
Long, to affect the efficiency of image retrieval.
Summary of the invention
The method and device for being designed to provide a kind of image retrieval of the embodiment of the present invention, it is intended to solve in the prior art
Since picture number increases, the impacted problem of the efficiency of image retrieval.
The embodiments of the present invention are implemented as follows, a kind of method of image retrieval, comprising:
Obtain first description of every samples pictures and the second description of input picture；
By the first description and the second description respectively by Fisher vector form be converted into only comprising " 0 " and
The vector form of " 1 "；
The similarity of the second description and the first description after each conversion after calculating separately conversion, root
The samples pictures are screened according to calculated result, obtain the search result of the input picture.
The another object of the embodiment of the present invention is to provide a kind of device of image retrieval, comprising:
Acquiring unit, for obtaining first description of every samples pictures and the second description of input picture；
Converting unit, for converting the first description and the second description by Fisher vector form respectively
At the vector form for only including " 0 " and " 1 "；
Retrieval unit, for calculating separately the second description after converting and first description after each conversion
The similarity of son, screens the samples pictures according to calculated result, obtains the search result of the input picture.
It only includes element " 0 " and element " 1 " that the embodiment of the present invention, which is converted to description of picture by Fisher vector,
Vector reduces the sample for carrying out inner product of vectors operation to carry out preliminary screening to picture before carrying out inner product of vectors operation
Picture number improves inspection to greatly improve the arithmetic speed of image retrieval in large-scale picture retrieval system
Rope efficiency.
Detailed description of the invention
Fig. 1 is the implementation flow chart of the method for image retrieval provided in an embodiment of the present invention；
Fig. 2 is the specific implementation flow chart of the method S102 of image retrieval provided in an embodiment of the present invention；
Fig. 3 is the specific implementation flow chart of the method S103 of image retrieval provided in an embodiment of the present invention；
Fig. 4 is the structural block diagram of the device of image retrieval provided in an embodiment of the present invention.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
Fig. 1 shows the implementation process of the method for image retrieval provided in an embodiment of the present invention, and details are as follows:
In S101, first description of every samples pictures and the second description of input picture are obtained.
Firstly, successively carrying out feature extraction and feature clustering respectively to samples pictures and input picture based on BOW model, obtain
To description of every picture, which is indicated with Fisher vector form.
Wherein it is possible to by the way of feature point extraction, for example, scale invariant feature converts (Scale-invariant
Feature transform, SIFT) algorithm, SURF (Speeded Up Robust Features) algorithm etc., to input picture
Carry out feature extraction, wherein SIFT feature has invariable rotary shape, scale invariability and maintains the invariance to brightness change
Equal superperformances.Simultaneously as picture color information also has certain discrimination, accordingly it is also possible to carry out to input picture
Color feature extracted.Further, it is also possible to carry out the direction gradient histogram of whole image or a certain parts of images to input picture
Scheme (Histograms of Oriented Gradients, HOG) feature extraction.
In the present embodiment, using GMM, a certain number of Gaussian functions are chosen, utilize the characteristic point training GMM extracted
Parameters.Wherein, with X={ xt, t=1 ..., T } and indicate that the feature vector extracted from picture, the parameter of GMM are λ
={ wi, μi, ∑i, i=1 ..., N }, wi、μiWeight, mean vector and the covariance matrix of Gaussian function i are respectively indicated with ∑ i,
The N is the number of Gaussian function.
And the Fisher vector based on GMM generates in many research work mutually it shall be mentioned that according to database picture (i.e. sample
This picture) scale, select GMM quantity N appropriate, for image feature extraction generate D dimension feature vector (for example,
SIFT feature vector be 128 dimension, SURF feature vector be 64 dimension, color feature vector be 96 dimension), the Fisher ultimately produced to
Amount dimension is (2*D+1) * N-1, and generally in practical applications, due to having ignored 0 rank information, last practical vector dimension
Degree is 2*D*N.
From the above, it can be seen that the growth of picture number can lead to the growth of GMM quantity N indirectly, and for advising greatly
The dimension of the image retrieval of mould, usually last vector is dimension up to ten thousand, using the meter of inner product of vectors when calculating picture similarity
Calculation mode, so undoubtedly can be excessively high because of dimension, and picture number is excessive and substantially reduces the efficiency of operation.Therefore, in S102,
Fisher vector is converted, to simplify the similarity calculation complexity of picture:
In S102, the first description and the second description are converted into only by Fisher vector form respectively
Vector form comprising " 0 " and " 1 ".
As an embodiment of the present invention, Fisher vector can be converted into same dimension only comprising element " 0 " and
The vector (hereinafter referred to as 01 vector) of " 1 ", conversion method is as follows:
The element for being greater than or equal to 0 in the Fisher vector is set 1, and by the member in the Fisher vector less than 0
Element sets 0, obtains third description of every samples pictures and the 4th description of the input picture.
Specifically, Fisher vector description is reset by following formula:
F (z)=(sign (z)+1)/2 (z ≠ 0)
For z=0, we directly enable f (z)=1, are handled by above formula, can generate and Fisher vector is the same as dimension
01 vector, the third that the first description of every samples pictures Fisher vector form is converted into 01 vector form as a result, are retouched
Son is stated, the second description of input picture Fisher vector form is converted into the 4th description of 01 vector form.
From the perspective of statistics, to a certain extent, one group of data for being distributed similar equal length should have
Similar mean value therefore further, can also be by method shown in Fig. 2 as another embodiment of the present invention, will
Fisher vector is converted to lower 01 vector of dimension:
In S201, the Fisher vector is divided into M parts, the M is the integer greater than 1.
In S202, mean value is taken to every portion Fisher vector.
In S203, if the mean value be greater than or equal to 0, set 1 in the mean value position, if the mean value less than 0,
0 then is set in the mean value position, the 5th description of every samples pictures is obtained and the 6th of the input picture retouches
State son.
The first description of every samples pictures Fisher vector form is converted into the 5th of 01 vector form and retouches as a result,
Son is stated, the second description of input picture Fisher vector form is converted into the 6th description of 01 vector form, and the 6th
The vector dimension for describing son will be lower than the vector dimension of the 4th description.
It should be noted that Fisher vector to be converted into 01 vector of same dimension, and Fisher vector is converted into
01 vector of more low dimensional, both conversion processes can carry out simultaneously, regardless of order can also successively carry out.
For the M dimensional vector of generation, several pieces can be classified as, each piece is safeguarded in a manner of establishing table, in this way
In subsequent calculating process, amount of calculation can be reduced by tabling look-up.
In S103, the second description after calculating separately conversion and the first description after each conversion
Similarity screens the samples pictures according to calculated result, obtains the search result of the input picture.
After being converted Fisher vector by S102, samples pictures are carried out based on the vector after conversion preliminary
Screening, filtering reduce the quantity of samples pictures, then remaining samples pictures and input picture are carried out inner product of vectors, thus
To search result.
Preferably, when Fisher vector is converted to 01 vector of 01 vector sum more low dimensional of same dimension,
The implementation procedure of S103 is as shown in Figure 3:
In S301, the 6th description and the 5th description of every samples pictures of the input picture are calculated separately
First similarity of son, the samples pictures low to first similarity are filtered.
Firstly, than the similarity between 01 vector to more low dimensional.For sample graph of the number in terms of 100,000 or even million
For piece, the inner product operation speed of vector of the inner product operation speed of low dimensional than directly carrying out dimensions up to ten thousand is run much faster,
Preferably, the M can be 64, to keep balance preferably between arithmetic speed and filtering accuracy.According to inner product operation
As a result, calculating the first similarity between the 6th description and the 5th description of different samples pictures, and similar by first
Degree is filtered lower than the samples pictures of preset threshold.
In S302, the 4th description and filtered every samples pictures of the input picture are calculated separately
Second similarity of third description, it is for re-filtering to the samples pictures that second similarity is low.
After being filtered by S301 to samples pictures, to similar with the 01 vector progress of dimension to Fisher vector
Degree calculates, that is, the 4th description for inputting picture and third description of filtered every samples pictures are carried out similarity
It calculates.Specifically, step-by-step exclusive or, the number for the position for being 1 in statistical calculation result can be carried out to 01 vector of the two, number is got over
It is more, think that the similarity of two width pictures is higher, using be in operation result 1 position number as the second similarity, and by second
Similarity is for re-filtering lower than the samples pictures of preset threshold.
In S303, described respectively to the second of the input picture with filtered every samples pictures again
First describe son and do inner product of vectors, obtain the input picture match search result.
By S302 it is for re-filtering to samples pictures and then with remaining samples pictures few in number it is direct
Inner product of vectors is carried out with input picture, and the optimal samples pictures of calculated result are matched as the search result of the input picture.
In the present embodiment, be experimentally confirmed, carry out 64 bit arithmetic the time it takes be it is shortest, carry out with
Fisher vector with the step-by-step XOR operation of 01 vector of dimension time secondly, directly carry out inner product of vectors spent operation when
Between longest.Although will increase additional sort operation using the screening process of S301 and S302 twice, in contrast to only carrying out
The inner product operation of vector, with the growth of samples pictures quantity, shadow of the time spent by sort operation to final operation time
The effect of sound just substantially reduces, and therefore, generally speaking, for large-scale picture retrieval, this programme can be to a certain degree
Upper improvement recall precision, and the results show this programme can bring the slightly raising of retrieval precision.
It only includes element " 0 " and element " 1 " that the embodiment of the present invention, which is converted to description of picture by Fisher vector,
Vector reduces the sample for carrying out inner product of vectors operation to carry out preliminary screening to picture before carrying out inner product of vectors operation
Picture number improves inspection to greatly improve the arithmetic speed of image retrieval in large-scale picture retrieval system
Rope efficiency.
Corresponding to the method for image retrieval described in foregoing embodiments, Fig. 4 shows image provided in an embodiment of the present invention
The structural block diagram of the device of retrieval, for ease of description, only the parts related to this embodiment are shown.
Referring to Fig. 4, which includes:
Acquiring unit 41 obtains first description of every samples pictures and the second description of input picture.
First description and the second description are converted by Fisher vector form by converting unit 42 respectively
It only include the vector form of " 0 " and " 1 ".
Retrieval unit 43, the second description after calculating separately conversion and the first description after each conversion
Similarity, the samples pictures are screened according to calculated result, obtain it is described input picture search result.
Optionally, the converting unit 42 is specifically used for:
The element for being greater than or equal to 0 in the Fisher vector is set 1, and by the member in the Fisher vector less than 0
Element sets 0, obtains third description of every samples pictures and the 4th description of the input picture.
Optionally, the converting unit 42 is also used to:
The Fisher vector is divided into M parts, the M is the integer greater than 1.
Mean value is taken to every portion Fisher vector.
If the mean value is greater than or equal to 0,1 is set in the mean value position, it is equal at this if the mean value is less than 0
Value position sets 0, obtains the 5th description of every samples pictures and the 6th description of the input picture.
Optionally, the M is 64.
Optionally, the retrieval unit 43 includes:
First filtering subelement calculates separately the of the 6th description of the input picture and every samples pictures
First similarity of five description, the samples pictures low to first similarity are filtered.
Second filtering subelement calculates separately the 4th description and the filtered every sample of the input picture
Second similarity of third description of picture, it is for re-filtering to the samples pictures that second similarity is low.
Third filters subelement, describes respectively to the second of the input picture with the filtered every sample again
The first of this picture describes son and does inner product of vectors, obtains the input picture match search result.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in essence of the invention
Made any modifications, equivalent replacements, and improvements etc., should all be included in the protection scope of the present invention within mind and principle.
Claims (4)
1. a kind of method of image retrieval characterized by comprising
Obtain first description of every samples pictures and the second description of input picture；
First description and the second description are converted into only by Fisher vector form comprising " 0 " and " 1 " respectively
Vector form；
The similarity of the second description and the first description after each conversion after calculating separately conversion, according to meter
It calculates result to screen the samples pictures, obtains the search result of the input picture；
It is described by the first description and the second description respectively by Fisher vector form be converted into only comprising " 0 " and
The vector form of " 1 " includes:
The element for being greater than or equal to 0 in the Fisher vector is set 1, and the element in the Fisher vector less than 0 is set
0, obtain third description of every samples pictures and the 4th description of the input picture；
The Fisher vector is divided into M parts, the M is the integer greater than 1；
Mean value is taken to every portion Fisher vector；
If the mean value be greater than or equal to 0, set 1 in the mean value position, if the mean value less than 0, in the mean value institute
0 is set in position, obtains the 5th description of every samples pictures and the 6th description of the input picture；
The similarity for calculating separately the second description and the first description after each conversion after conversion, root
The samples pictures are screened according to calculated result, the search result for obtaining the input picture includes:
Calculate separately the 6th description of the input picture and the 5th description of samples pictures described in every first is similar
Degree, the samples pictures low to first similarity are filtered；
The third of the 4th description and the filtered every samples pictures that calculate separately the input picture describes son
Second similarity, it is for re-filtering to the samples pictures that second similarity is low；
Sub the first description with the filtered every samples pictures again is described to the second of the input picture respectively
Inner product of vectors is done, the input picture match search result is obtained.
2. the method as described in claim 1, which is characterized in that the M is 64.
3. a kind of device of image retrieval characterized by comprising
Acquiring unit, for obtaining first description of every samples pictures and the second description of input picture；
Converting unit, for the first description and the second description to be converted into only by Fisher vector form respectively
Vector form comprising " 0 " and " 1 "；
Retrieval unit, for the first description after calculating separately the second description after conversion with each conversion
Similarity screens the samples pictures according to calculated result, obtains the search result of the input picture；
The converting unit is specifically used for:
The element for being greater than or equal to 0 in the Fisher vector is set 1, and the element in the Fisher vector less than 0 is set
0, obtain third description of every samples pictures and the 4th description of the input picture；
The Fisher vector is divided into M parts, the M is the integer greater than 1；
Mean value is taken to every portion Fisher vector；
If the mean value be greater than or equal to 0, set 1 in the mean value position, if the mean value less than 0, in the mean value institute
0 is set in position, obtains the 5th description of every samples pictures and the 6th description of the input picture；
The retrieval unit includes:
First filtering subelement, for calculating separately the of the 6th description of the input picture and every samples pictures
First similarity of five description, the samples pictures low to first similarity are filtered；
Second filtering subelement, for calculating separately the 4th description and the filtered every sample of the input picture
Second similarity of third description of picture, it is for re-filtering to the samples pictures that second similarity is low；
Third filters subelement, describes for respectively to the second of the input picture with the filtered every sample again
The first of this picture describes son and does inner product of vectors, obtains the input picture match search result.
4. device as claimed in claim 3, which is characterized in that the M is 64.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510083238.XA CN105989128B (en) | 2015-02-13 | 2015-02-13 | A kind of method and device of image retrieval |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510083238.XA CN105989128B (en) | 2015-02-13 | 2015-02-13 | A kind of method and device of image retrieval |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105989128A CN105989128A (en) | 2016-10-05 |
| CN105989128B true CN105989128B (en) | 2019-05-07 |
Family
ID=57042577
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510083238.XA Active CN105989128B (en) | 2015-02-13 | 2015-02-13 | A kind of method and device of image retrieval |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105989128B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102819582A (en) * | 2012-07-26 | 2012-12-12 | 华数传媒网络有限公司 | Quick searching method for mass images |
| CN102982561A (en) * | 2012-12-31 | 2013-03-20 | 哈尔滨工业大学 | Method for detecting binary robust scale invariable feature of color of color image |
| CN104268602A (en) * | 2014-10-14 | 2015-01-07 | 大连理工大学 | Shielded workpiece identifying method and device based on binary system feature matching |
| CN104282005A (en) * | 2014-09-19 | 2015-01-14 | 天津航天中为数据系统科技有限公司 | Video image stitching method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2397424A1 (en) * | 2002-08-09 | 2004-02-09 | Mohammed Lamine Kherfi | Content-based image retrieval using positive and negative examples |
-
2015
- 2015-02-13 CN CN201510083238.XA patent/CN105989128B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102819582A (en) * | 2012-07-26 | 2012-12-12 | 华数传媒网络有限公司 | Quick searching method for mass images |
| CN102982561A (en) * | 2012-12-31 | 2013-03-20 | 哈尔滨工业大学 | Method for detecting binary robust scale invariable feature of color of color image |
| CN104282005A (en) * | 2014-09-19 | 2015-01-14 | 天津航天中为数据系统科技有限公司 | Video image stitching method and device |
| CN104268602A (en) * | 2014-10-14 | 2015-01-07 | 大连理工大学 | Shielded workpiece identifying method and device based on binary system feature matching |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105989128A (en) | 2016-10-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bianco et al. | Combination of video change detection algorithms by genetic programming | |
| Uijlings et al. | Video classification with densely extracted hog/hof/mbh features: an evaluation of the accuracy/computational efficiency trade-off | |
| JP5588165B2 (en) | Image processing apparatus, image processing method, and program | |
| CN105224951B (en) | A kind of vehicle type classification method and sorter | |
| Grana et al. | A fast approach for integrating ORB descriptors in the bag of words model | |
| CN109377445A (en) | Model training method, the method, apparatus and electronic system for replacing image background | |
| JP6897749B2 (en) | Learning methods, learning systems, and learning programs | |
| CN110598019B (en) | Repeated image identification method and device | |
| CN111401099B (en) | Text recognition method, device and storage medium | |
| CN107430604A (en) | The semantic expressiveness of picture material | |
| CN110532950B (en) | Video feature extraction method and micro-expression identification method based on micro-expression video | |
| CN109344869A (en) | A kind of disaggregated model optimization method, device and storage equipment, program product | |
| CN110110113A (en) | Image search method, system and electronic device | |
| CN114169381A (en) | Image annotation method and device, terminal equipment and storage medium | |
| Nanni et al. | How could a subcellular image, or a painting by Van Gogh, be similar to a great white shark or to a pizza? | |
| Bharath et al. | Scalable scene understanding using saliency-guided object localization | |
| CN111797971A (en) | Method, device and electronic system for processing data by using convolutional neural network | |
| CN105095468A (en) | Novel image retrieval method and system | |
| CN104966109A (en) | Medical laboratory report image classification method and apparatus | |
| CN113536020B (en) | Method, storage medium and computer program product for data query | |
| CN111783812A (en) | Method and device for identifying forbidden images and computer readable storage medium | |
| JP2014115920A (en) | Multi-class identifier, method, and program | |
| CN105488529A (en) | Identification method and apparatus for source camera model of picture | |
| CN109299295B (en) | Blue printing layout database searching method | |
| Tang et al. | Detection of various image operations based on CNN |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |