CN108280187A - A kind of classification image search method based on convolutional neural networks depth characteristic - Google Patents
A kind of classification image search method based on convolutional neural networks depth characteristic Download PDFInfo
- Publication number
- CN108280187A CN108280187A CN201810066649.1A CN201810066649A CN108280187A CN 108280187 A CN108280187 A CN 108280187A CN 201810066649 A CN201810066649 A CN 201810066649A CN 108280187 A CN108280187 A CN 108280187A
- Authority
- CN
- China
- Prior art keywords
- image
- feature
- similarity
- vector
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013527 convolutional neural network Methods 0.000 title claims abstract description 30
- 238000000034 method Methods 0.000 title claims abstract description 30
- 238000000605 extraction Methods 0.000 claims abstract description 17
- 238000012549 training Methods 0.000 claims abstract description 9
- 230000004927 fusion Effects 0.000 claims abstract description 3
- 239000013598 vector Substances 0.000 claims description 49
- 229910002056 binary alloy Inorganic materials 0.000 claims description 10
- 239000000284 extract Substances 0.000 claims description 10
- 239000000203 mixture Substances 0.000 claims description 4
- 238000000691 measurement method Methods 0.000 claims description 2
- 238000013526 transfer learning Methods 0.000 claims description 2
- 238000005259 measurement Methods 0.000 claims 1
- 230000006835 compression Effects 0.000 abstract description 13
- 238000007906 compression Methods 0.000 abstract description 13
- 238000004364 calculation method Methods 0.000 abstract description 2
- 238000012512 characterization method Methods 0.000 description 8
- 230000004913 activation Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 238000010606 normalization Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000007500 overflow downdraw method Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000003252 repetitive effect Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000003475 lamination Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 230000007935 neutral effect Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
- G06F16/5838—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The present invention provides a kind of classification image search method based on convolutional neural networks depth characteristic.Basic principle is that first, convolutional neural networks of the training for feature extraction determine network parameter；Then, characteristics of image is extracted using trained convolutional neural networks, multiple convolutional layer binary features and a full articulamentum binary features can be obtained；Secondly, multiple convolutional layer binary features are used for primary dcreening operation retrieval phase, further carry out multi-feature similarity fusion after compression, filter out candidate image collection, reduces range of search；Finally, precise search is carried out on candidate image collection using full articulamentum binary features, obtain final retrieval result.Based on common image retrieve data set the experimental results showed that, compared to existing image search method, this method uses the characteristic manner of image more comprehensively, Feature Compression method is more simple and efficient, retrieval rate is higher, and the mode decentralized system calculation amount of grading search, is conducive to speed up to parallel, has practical value.
Description
Technical field
The invention belongs to image processing techniques and information retrieval field, it is related to a kind of utilizing convolutional Neural net in deep learning
Network extracts the classification image search method that depth characteristic is realized.
Background technology
Increase instantly in image data explosion, it is management large nuber of images number to carry out quickly and effectively retrieval to image data
According to a kind of important mode, content-based image retrieval (Content-Based Image Retrieval, abbreviation CBIR) skill
Art is come into being under such actual demand.CBIR is that one kind realizing matched image by extracting image content information
Retrieval mode, target are：User gives a width query image, from a large scale database quick-searching to query graph
As the related similar image of content, and user is returned to according to sequencing of similarity.
Traditional CBIR systems are by the visual signature in manual extraction image, such as color, texture, shape, local feature
The features such as Aggregation Descriptor (Vector of Locally Aggregated Descriptors, VLAD) realize image retrieval
Function.There is also some limitations for these features manually extracted：On the one hand, for different types of image, different characteristic
Validity it is different, when being applied on large-scale data set, generalization ability is poor；On the other hand, these features belong to vision
Shallow-layer feature, characterization be imaging surface low-level information, can not reflect the contents semantic information of image, exist and user manage
Solution there are problems that inconsistent " semantic gap ", it is difficult to accurately express picture material.
In recent years with the rise of deep learning, convolutional neural networks (Convolutional Neural Network,
CNN) it has been proved to have big advantage in terms of visual characteristic, the further feature of CNN is closer to it is appreciated that level solves
Picture material is released, the limitation of conventional method image feature representation has been broken.Existing CBIR systems are using CNN as feature extraction
Device characterizes image by extracting the last full articulamentum feature of network, achieves preferable retrieval effectiveness.CNN is more than one
The network structure of hidden layer, the convolutional layer among network equally have prodigious potentiality in characterization image information, some deep layers
The characterization ability of convolutional layer has been even more than full articulamentum, and existing CNN image search methods only extract full articulamentum feature, and
The effect for having ignored convolutional layer feature fails to make full use of CNN characteristic informations, results in the waste of image information in convolutional layer.
Although the feature ability to express in CNN is stronger, have the characteristics that dimension is higher, for large scale database,
All images directly are characterized using the feature vector extracted, need great storage resource and matched computing cost, it is difficult to
Meet Search Requirement.Therefore particularly significant to Feature Compression processing, under the premise of ensureing that effective information is not suffered a loss, maximum journey
The removal feature vector redundancy of degree, compressive features dimension.Existing Feature Compression method common are principal component (Principal
Component Analysis, PCA) dimensionality reduction, Hash coding is such as local sensitivity Hash (Locality-Sensitive
Hashing, LSH), the modes such as semantic Hash (Semantic Hashing, SH), these methods be commonly available to one-dimensional characteristic to
When measuring, and being applied to two dimension structure feature, being directly translated into one-dimensional characteristic is handled, and a part of two dimensional character is had lost
Structural information, and more additional step is needed to realize squeeze operation, increases system-computed amount and algorithm complexity.
In conclusion the Efficient Characterization of image is the key that determine CBIR system retrieval performances.Existing CBIR method bases
Feature is extracted in CNN, although the existing improvement in terms of characterization image, there are still CNN characteristic uses are insufficient, Feature Compression
This body structure is detached from algorithm with feature, and the higher problem of complexity.
Invention content
Characterization image is not goed deep into solve tradition CBIR systems, the feature of extraction is in the presence of " semantic gap " and existing
CBIR systems based on CNN are not suitable with two dimensional character and algorithm complexity to network characterization using insufficient, Feature Compression algorithm
Higher problem, the present invention propose a kind of image search method based on CNN depth characteristics.
The technical scheme is that：Using the first screening of a kind of image search method of two-stage mechanism, including the first order
The precise search of rope and the second level.This method extracts the further feature of image based on CNN, and the feature of the different layers extracted is transported
It uses in retrieval not at the same level, the linear unit R ectified Linear Units of the amendment from convolution module and full link block
(ReLU) class binary features figure is extracted after layer, converts to obtain the binary features figure vector of multilayer, maximum journey by binary system
Degree utilizes multilayer feature.Wherein, the binary features figure vector of convolution module is used for the preliminary screening of the first order, will be connected entirely
The binary features figure vector of module is used for the precise search of the second level.Further feature combines the search mechanism of classification, is ensureing
While retrieval rate, retrieval rate is promoted, realizes the quick precise search in large-scale image library.
The step of the present invention is as follows：
The first step：The parameter setting of feature extraction network：
Using CNN as feature extraction network, network includes the full articulamentum module of multiple convolution modules and one.Network parameter
The process of setting is：The network is carried out to classification pre-training on large database concept first to determine suitable network initial parameter；
Then transfer learning is carried out again, training trim network parameter, makes it be showed on target data set optimal on target image library,
Complete the determination of feature extraction network parameter.
Second step：Extract the binary system depth characteristic of image：
Image is inputted into trained CNN, extracts class two after the ReLU layers of last convolution module and full link block respectively
System further feature, using ReLU by left-half zero setting, the binary system that right half part remains unchanged activates feature, can be direct
Class binary features figure vector is obtained from network.It indicates from k-th of ReLU layers of class two extracted
System characteristic pattern, the layer have N number of characteristic pattern, thereforeVector has N number of element, each element Vi k(i=1 ..., N) table
A m*m characteristic pattern is levied, the element in all characteristic patterns is all non-negative after ReLU activation primitives.(note：Characteristic pattern herein
Size m*m depend on the characteristic pattern sizes that are arranged in this layer of CNN)
By by all Vi kNonzero element in (i=1 ..., N) characteristic pattern sets 1, can obtain by N number of size being m*m marks
Quasi- binary features figureThe normal binary characteristic pattern vector of compositionThis step
Extract the full connection binary features figure vector of n convolution binary features figure vector sum one in convolution module.
Third walks：Primary dcreening operation retrieval phase：
N kind of convolution binary features figure vectors are used for the first order primary dcreening operation stage, are usedK=1,2,
3 ..., n is indicated respectively.In order to realize quick-searching, each characteristic pattern in each characteristic pattern vector is subjected to sum operation：Obtain compressed feature vector： Respectively
Indicate n kind convolution feature vectors.The phase of image and query image in target image library is measured with this n kind convolution feature vector respectively
Like degree, corresponding similarity sequence is obtained T is of image in target image library
Number.Similarity fusion method is recycled, the similarity obtained based on n feature is fused into final global similarity Sim, it will
Sim sorts from high to low, takes image composition candidate image library P={ Is of the global similarity Sim more than threshold value Th1,I2,...,
IM}。
The measurement method of similarity is：
If the feature vector of query imageWith the feature vector of image in target image libraryM is the size of primitive character figure.If the initial similarity S of two feature vectors is 0, for
Image in every characteristics of image library, seeks F(q)And F(t)The absolute difference sub of middle corresponding element, judges the range of difference sub,
Similarity is changed successively according to following rules：IfThen S=S+3；If m/2<sub<M, then S=S+2；If m
≤sub<2m, then S=S+1；If 2m≤sub<3m, then S=S-1；If 3m≤sub<4m, then S=S-2；If sub>4m, then S=
S-3.It can obtain the similarity sequence that all images are obtained based on k-th of feature in target image libraryT
Indicate the total number of image in target image library.
Similarity fusion method be：
Min-max first normalizes three kinds of similarity sequences：
Similarity sequence after being normalizedDue under the similarity sequence curve after sequence
Area and its feature retrieval performance there are inverse correlation relationship, the similarity fusion weight for calculating k-th of feature is：
Wherein
Global similarity after can finally obtaining that target tightening and being merged between image t and query image q is：
4th step：The precise search stage：
In order to further enhance retrieval rate, the candidate image library P={ I that are retrieved based on previous step1,I2,...,IM,
The retrieval of this stage measures similarity size using full connection vector of binary features by Hamming distance：Sim (q, t)=N-H (q,
T), wherein N is the total length of full connection features vector, Hamming distances of the H (q, t) between target image t and query image q.
By candidate image library according to similarity size minor sort again, final retrieval result is obtained.
The beneficial effects of the invention are as follows：
The present invention utilizes multilayer neural network further feature, realizes a kind of search mechanism of classification.With existing search method phase
Than following advantageous effect can be obtained：
(1) the further feature more adjunction extracted using deep neural network compared to the shallow-layer feature of manual extraction, the present invention
It is bordering on the semantic understanding of people, across the obstacle of " semantic gap ", the content information of image is preferably characterized, significantly enhances inspection
Rope accuracy rate.
(2) present invention completes the compression binaryzation work of feature using the nonlinear activation function in neural network, and utilization is non-thread
Property the binary system activation feature that has of activation primitive itself, realize and the binary system of complex characteristic converted, avoid existing feature
The higher compressed encoding operation of complexity in processing method；In compression processing convolution two dimension structure feature, with two dimensional character figure
It is operated for unit, avoids the occurrence of the case where being detached from two-dimensional structure.
(3) present invention uses grading search mechanism, introduces the convolutional layer feature of deep layer in first order retrieval, utilizes multilayer depth
Layer convolution feature filters out candidate target collection in extensive target data concentration, destination number is reduced for second level retrieval.It compares
CNN characteristic informations can be utilized to the greatest extent in the method for the existing search method based on CNN, this grading search, carried
While rising retrieval accuracy, decentralized system calculation amount is conducive to utilize the tall and handsome universal parallel framework (Compute up to company
Unified Device Architecture, CUDA) it speeds up to parallel.
Description of the drawings
Fig. 1 is the image indexing system overview flow chart of the present invention；
Fig. 2 is VGG depth convolutional neural networks structural schematic diagram of the present invention；
Fig. 3 is present invention retrieval first stage measuring similarity algorithm pattern；
Fig. 4 is the normalization similarity curve relational graph based on different characteristic；
Specific implementation mode
Fig. 1 is the image indexing system overview flow chart of the present invention.Image retrieval flow is divided into four steps：
The first step, the parameter setting of feature extraction network：
Using the deeper VGG network architectures of the number of plies in CNN as feature extraction network.
Fig. 2 is the schematic network structure of VGG networks.
VGG networks use more hidden layer configurations, classify to the image of input, by an input layer by size be 224*224
Triple channel image input network, by five convolution modules and a full articulamentum module extraction characteristics of image, finally utilize
These features export the probability to all categories in output layer.Wherein first four convolution module is using monovolume lamination and ReLU layers
Structure, the 5th convolution module include three convolutional layers and ReLU layers of structure, realize cubic convolution operation, realize further feature
Extraction, due to further feature picture material characterization on more advantage, by the convolutional layer conv5_1 of the 5th convolution module,
The feature of conv5_2, conv5_3 and full articulamentum fc7 are for the present invention.
The setting up procedure of VGG network parameters is as follows：
1) by the network, in large data sets ImageNet, (ImageNet data sets comprise more than 1,200,000 images, contain 1000 altogether
Class) on carry out classification pre-training and determine suitable network initial parameter.It is high to initialize network weight parameter W~(0~0.01)
This distribution, initialization network biasing bias is 0, and initial learning rate is set as 0.001, subtracts learning rate 100 times per iteration
10 times small, repetitive exercise is until network losses function convergence；
2) the feature extraction network after pre-training is finely tuned on target image library, it is real according to target database trim network parameter
Now complete the determination of feature extraction network parameter.Using the good network parameter of previous step pre-training as the initial parameter of this step,
Reduce learning rate to 1e-5, finely tunes training whole network, repetitive exercise completes feature extraction until network losses function convergence
The preparation of network.
Second step extracts the binary system depth characteristic in target image library and query image：
Target image library and query image are inputted into the trained VGG networks of previous step, the convolutional layer in the 5th convolution module
The ReLU layers of fc7 extract class binary system in ReLU layers and last full link block after conv5_1, conv5_2, conv5_3
Further feature figure vectorBy by all characteristic pattern V in the vectori kIt is non-in (i=1 ..., N)
Neutral element sets 1, is converted to normal binary characteristic pattern vector3 volumes are obtained for every image
The full connection binary features figure vector of product binary features figure 1 full articulamentum of vector sum.
Third walks, primary dcreening operation retrieval phase：
(1) by 3 kinds of convolution binary features figure vectorsFor first order primary dcreening operation rank
Section.Each characteristic pattern in each vector is subjected to sum operation：Obtain compressed spy
Sign vector：
(2) similarity for measuring image and query image in target image library with these three convolution feature vectors respectively, obtains three
The similarity sequence in kind target image library and query image
Fig. 3 is primary dcreening operation retrieval phase measuring similarity algorithm pattern of the present invention.
The feature vector of input inquiry image qWith the feature of image t in target image library to
AmountThe characteristic pattern size m of convolutional layer conv5_1, conv5_2, conv5_3 are 7, N 512.
Initial similarity S is 0, for every image in target image library, acquires F(q)And F(t)The absolute difference of middle corresponding element
Sub changes similarity value according to the range of difference.Export phase of all images based on these three convolution features in target image library
Like degree series
(3) the obtained similarity of three kinds of features is fused into final global similarity Sim.
Fig. 4 is the normalization similarity curve figure based on different characteristic.
Five kinds of features based on conv3, conv4, conv5_1, conv5_2, conv5_3 convolutional layer in VGG networks calculate separately
Similarity sequence is obtained, all similarity sequences are obtained into the song of Fig. 4 after min-max is normalized by high sequence on earth
Line relational graph, abscissa are the similarity serial numbers after rearrangement, and ordinate is the similarity value after normalization, uses AP
(Average precision) weighs the retrieval performance of different characteristic, and the legend in the upper right corner reflects the AP values of each curvilinear characteristic,
The corresponding feature of different linearity curves in Fig. 4 legends from top to bottom is successively：conv4、conv5_1、conv3、conv5_3、
The feature of conv5_2 convolutional layers, AP values are higher, and reflection characteristic key performance is better.The curved line relation illustrates that retrieval effectiveness is got over
Good, the feature number of plies is deeper, and normalization similarity curve is more close to reference axis, and area under a curve is smaller, thus by different spies
The independent retrieval effectiveness of the area and feature under normalized curve is levied into inverse correlation relationship, the similarity power of different characteristic is set
Value.
(4) image composition candidate image library P={ Is of the Sim more than threshold value Th=0.5 is taken1,I2,...,IM}。
4th step：The precise search stage：
The binary features figure vector of full connection fc7 is used for the precise search of second stage.Based on candidate image library P={ I1,
I2,...,IM, the retrieval of this stage obtains similarity sim by the Hamming distance H (q, t) between target image t and query image q
(q, t)=N-H (q, t), wherein N=4096 are the length of fc7 feature vectors.By the image in Candidate Set according to similarity by big
To small sequence, final retrieval result is obtained.
The image search method that the present invention designs is compared with other search methods.Tables 1 and 2 is to be based on opening respectively
The Feature Compression method comparing result that data set Inria Holidays and Oxford Buildings are tested is put, is examined in table
It can be measured without hesitation with Average Accuracy AP.On two kinds of Catalog Search test sets, Feature Compression method of the invention exists
It is showed in various features well, the accuracy rate retrieved compared with other six kinds of common Feature Compression methods is higher.
We also by the image indexing system of the present invention and traditional searching system based on VLAD features, be based on connecting entirely
Retrieval effectiveness is preferably based on the full connection after summation down-sampling and the operation of PCA dimensionality reductions in the searching system and CNN of layer feature
The searching system of layer feature compares, and the results are shown in Table 3.The present invention is right in the case where not increasing system complexity
The accuracy rate of retrieval has a certain upgrade, and compared with several comparison searching systems in table, retrieval rate is higher, and advantage is bright
It is aobvious.
Feature Compression method contrast table of the table 1 based on Inria Holidays data sets
Feature Compression method contrast table of the table 2 based on Oxford Buildings data sets
3 different images searching system retrieval rate contrast table of table
Claims (1)
1. a kind of classification image search method based on convolutional neural networks depth characteristic, which is characterized in that include the following steps:
The first step：The parameter setting of feature extraction network：
Using convolutional neural networks as feature extraction network, network parameter is arranged using the method for transfer learning：
(1) network is carried out to classification pre-training on large database concept to determine suitable network initial parameter；
(2) the training trim network parameter on target image library, makes it be showed on target data set optimal, completes feature and carries
Take the determination of network parameter；
Second step：Extract the binary system depth characteristic of image：
(1) class vector of binary features is extracted from network：Image is inputted into trained convolutional neural networks, respectively most
Class is extracted after linear unit R ectified Linear Units (ReLU) layer of amendment of convolution module and full link block afterwards
Binary system further feature；If being from k-th of ReLU layers of class binary features figure extractedThe layer
There is N number of characteristic pattern, if each characteristic pattern Vi kThe size of (i=1 ..., N) is m*m；
(2) binaryzation feature vector：By by all Vi kNonzero element in (i=1 ..., N) characteristic pattern sets 1, will be each
Vi k(i=1 ..., N) it is converted into normal binary characteristic patternObtain normal binary characteristic pattern vectorExtract the full connection binary system of n convolution binary features figure vector sum one in convolution module
Characteristic pattern vector；
Third walks：Primary dcreening operation retrieval phase：
N convolution binary features figure vector is used for the first order primary dcreening operation stage, is usedK=1,2 ..., n
It indicates respectively；
(1) feature vector is compressed：Each characteristic pattern in each convolution feature vector is subjected to sum operation：Obtain compressed feature vector：
(2) similarity measurement：The phase of image and query image in target image library is measured with this n kind convolution feature vector respectively
Like degree, the measurement method of similarity is：
If the feature vector of query imageWith the feature vector of image in target image libraryM is the size of primitive character figure；If the initial similarity S of two feature vectors is 0；
(1) for the image in every target image library, F is sought(q)And F(t)Middle corresponding element absolute value of the difference sub；
(2) it is directed to each element difference sub of feature vector, similarity is changed successively according to following rules：
IfThen S=S+3；If m/2<sub<M, then S=S+2；
If m≤sub<2m, then S=S+1；If 2m≤sub<3m, then S=S-1；
If 3m≤sub<4m, then S=S-2；If sub>4m, then S=S -3；
Obtain the similarity sequence that all images are obtained based on k-th of feature in target image libraryT tables
Show the total number of image in target image library；
(3) multi-feature similarity merges：The obtained similarity of n kind features is fused into final global similarity Sim, phase
It is like the method that degree merges：
(1) min-max normalizes three kinds of similarity sequences：
Similarity sequence after being normalized
(2) since there are inverse correlation relationship, meters for the retrieval performance of similarity sequence area under a curve and its feature after sequence
The similarity fusion weight for calculating k-th feature is：
Wherein
(3) target tightening merged between image t and query image q after global similarity be：
Sim is sorted from high to low, takes image composition candidate image collection P={ Is of the global similarity Sim more than threshold value Th1,
I2,...,IM}；
4th step：The precise search stage：
Based on candidate image collection P={ I1,I2,...,IM, phase is measured by Hamming distance using full connection vector of binary features
Like degree size：Sim (q, t)=N-H (q, t), wherein N be full connection features vector total length, H (q, t) be target image t and
Hamming distance between query image q；Candidate image collection is sorted according to similarity size, obtains final retrieval result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810066649.1A CN108280187B (en) | 2018-01-24 | 2018-01-24 | Hierarchical image retrieval method based on depth features of convolutional neural network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810066649.1A CN108280187B (en) | 2018-01-24 | 2018-01-24 | Hierarchical image retrieval method based on depth features of convolutional neural network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108280187A true CN108280187A (en) | 2018-07-13 |
| CN108280187B CN108280187B (en) | 2021-06-01 |
Family
ID=62804798
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810066649.1A Active CN108280187B (en) | 2018-01-24 | 2018-01-24 | Hierarchical image retrieval method based on depth features of convolutional neural network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108280187B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109657082A (en) * | 2018-08-28 | 2019-04-19 | 武汉大学 | Remote sensing images multi-tag search method and system based on full convolutional neural networks |
| CN109712140A (en) * | 2019-01-02 | 2019-05-03 | 中楹青创科技有限公司 | Method and device of the training for the full link sort network of evaporating, emitting, dripping or leaking of liquid or gas detection |
| CN110069644A (en) * | 2019-04-24 | 2019-07-30 | 南京邮电大学 | A kind of compression domain large-scale image search method based on deep learning |
| CN110110748A (en) * | 2019-03-29 | 2019-08-09 | 广州思德医疗科技有限公司 | A kind of recognition methods of original image and device |
| CN111177446A (en) * | 2019-12-12 | 2020-05-19 | 苏州科技大学 | Method for searching footprint image |
| CN111325712A (en) * | 2020-01-20 | 2020-06-23 | 北京百度网讯科技有限公司 | Method and device for detecting image validity |
| CN112308102A (en) * | 2019-08-01 | 2021-02-02 | 北京易真学思教育科技有限公司 | Image similarity calculation method, calculation device, and storage medium |
| CN113349792A (en) * | 2021-05-31 | 2021-09-07 | 平安科技（深圳）有限公司 | Multi-lead electrocardiosignal-based classification method, device, equipment and medium |
| CN113886629A (en) * | 2021-12-09 | 2022-01-04 | 深圳行动派成长科技有限公司 | Course picture retrieval model establishing method |
| WO2022156284A1 (en) * | 2021-01-22 | 2022-07-28 | 深圳市商汤科技有限公司 | Retrieval method and apparatus, and electronic device |
| CN115129921A (en) * | 2022-06-30 | 2022-09-30 | 重庆紫光华山智安科技有限公司 | Picture retrieval method and device, electronic equipment and computer-readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7242802B2 (en) * | 2002-07-01 | 2007-07-10 | Xerox Corporation | Segmentation method and system for Multiple Raster Content (MRC) representation of documents |
| CN104679863A (en) * | 2015-02-28 | 2015-06-03 | 武汉烽火众智数字技术有限责任公司 | Method and system for searching images by images based on deep learning |
| CN104834748A (en) * | 2015-05-25 | 2015-08-12 | 中国科学院自动化研究所 | Image retrieval method utilizing deep semantic to rank hash codes |
| CN105631296A (en) * | 2015-12-30 | 2016-06-01 | 北京工业大学 | Design method of safety face verification system based on CNN (convolutional neural network) feature extractor |
| CN106682233A (en) * | 2017-01-16 | 2017-05-17 | 华侨大学 | Method for Hash image retrieval based on deep learning and local feature fusion |
| CN106778526A (en) * | 2016-11-28 | 2017-05-31 | 中通服公众信息产业股份有限公司 | A kind of extensive efficient face identification method based on Hamming distance |
| CN106997380A (en) * | 2017-03-21 | 2017-08-01 | 北京工业大学 | Imaging spectrum safe retrieving method based on DCGAN depth networks |
-
2018
- 2018-01-24 CN CN201810066649.1A patent/CN108280187B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7242802B2 (en) * | 2002-07-01 | 2007-07-10 | Xerox Corporation | Segmentation method and system for Multiple Raster Content (MRC) representation of documents |
| CN104679863A (en) * | 2015-02-28 | 2015-06-03 | 武汉烽火众智数字技术有限责任公司 | Method and system for searching images by images based on deep learning |
| CN104834748A (en) * | 2015-05-25 | 2015-08-12 | 中国科学院自动化研究所 | Image retrieval method utilizing deep semantic to rank hash codes |
| CN105631296A (en) * | 2015-12-30 | 2016-06-01 | 北京工业大学 | Design method of safety face verification system based on CNN (convolutional neural network) feature extractor |
| CN106778526A (en) * | 2016-11-28 | 2017-05-31 | 中通服公众信息产业股份有限公司 | A kind of extensive efficient face identification method based on Hamming distance |
| CN106682233A (en) * | 2017-01-16 | 2017-05-17 | 华侨大学 | Method for Hash image retrieval based on deep learning and local feature fusion |
| CN106997380A (en) * | 2017-03-21 | 2017-08-01 | 北京工业大学 | Imaging spectrum safe retrieving method based on DCGAN depth networks |
Non-Patent Citations (1)
| Title |
|---|
| 孙韶言: "基于深度学习表征的图像检索技术", 《中国博士学位论文全文数据库》 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109657082B (en) * | 2018-08-28 | 2022-11-29 | 武汉大学 | Remote sensing image multi-label retrieval method and system based on full convolution neural network |
| CN109657082A (en) * | 2018-08-28 | 2019-04-19 | 武汉大学 | Remote sensing images multi-tag search method and system based on full convolutional neural networks |
| CN109712140A (en) * | 2019-01-02 | 2019-05-03 | 中楹青创科技有限公司 | Method and device of the training for the full link sort network of evaporating, emitting, dripping or leaking of liquid or gas detection |
| CN110110748A (en) * | 2019-03-29 | 2019-08-09 | 广州思德医疗科技有限公司 | A kind of recognition methods of original image and device |
| CN110069644A (en) * | 2019-04-24 | 2019-07-30 | 南京邮电大学 | A kind of compression domain large-scale image search method based on deep learning |
| CN110069644B (en) * | 2019-04-24 | 2023-06-06 | 南京邮电大学 | Compressed domain large-scale image retrieval method based on deep learning |
| CN112308102A (en) * | 2019-08-01 | 2021-02-02 | 北京易真学思教育科技有限公司 | Image similarity calculation method, calculation device, and storage medium |
| CN111177446A (en) * | 2019-12-12 | 2020-05-19 | 苏州科技大学 | Method for searching footprint image |
| CN111177446B (en) * | 2019-12-12 | 2023-04-25 | 苏州科技大学 | Method for searching footprint image |
| CN111325712A (en) * | 2020-01-20 | 2020-06-23 | 北京百度网讯科技有限公司 | Method and device for detecting image validity |
| CN111325712B (en) * | 2020-01-20 | 2024-01-23 | 北京百度网讯科技有限公司 | Method and device for detecting image validity |
| WO2022156284A1 (en) * | 2021-01-22 | 2022-07-28 | 深圳市商汤科技有限公司 | Retrieval method and apparatus, and electronic device |
| CN113349792B (en) * | 2021-05-31 | 2022-10-11 | 平安科技（深圳）有限公司 | Method, apparatus, device and medium for classifying multi-lead electrocardiosignal |
| CN113349792A (en) * | 2021-05-31 | 2021-09-07 | 平安科技（深圳）有限公司 | Multi-lead electrocardiosignal-based classification method, device, equipment and medium |
| CN113886629A (en) * | 2021-12-09 | 2022-01-04 | 深圳行动派成长科技有限公司 | Course picture retrieval model establishing method |
| CN115129921A (en) * | 2022-06-30 | 2022-09-30 | 重庆紫光华山智安科技有限公司 | Picture retrieval method and device, electronic equipment and computer-readable storage medium |
| CN115129921B (en) * | 2022-06-30 | 2023-05-26 | 重庆紫光华山智安科技有限公司 | Picture retrieval method, apparatus, electronic device, and computer-readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108280187B (en) | 2021-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108280187A (en) | A kind of classification image search method based on convolutional neural networks depth characteristic | |
| CN111198959B (en) | Two-stage image retrieval method based on convolutional neural network | |
| Raza et al. | Correlated primary visual texton histogram features for content base image retrieval | |
| Shuai et al. | Fingerprint indexing based on composite set of reduced SIFT features | |
| CN103186538A (en) | Image classification method, image classification device, image retrieval method and image retrieval device | |
| CN108984642A (en) | A kind of PRINTED FABRIC image search method based on Hash coding | |
| Sugamya et al. | A CBIR classification using support vector machines | |
| Kaur et al. | A novel technique for content based image retrieval using color, texture and edge features | |
| Saad et al. | Image retrieval based on integration between YC b C r color histogram and shape feature | |
| Ahmed et al. | Deep image sensing and retrieval using suppression, scale spacing and division, interpolation and spatial color coordinates with bag of words for large and complex datasets | |
| Naeem et al. | Deep learned vectors’ formation using auto-correlation, scaling, and derivations with CNN for complex and huge image retrieval | |
| Chen et al. | Instance retrieval using region of interest based CNN features | |
| CN109933682A (en) | A kind of image Hash search method and system based on semanteme in conjunction with content information | |
| CN111177435A (en) | CBIR method based on improved PQ algorithm | |
| CN110110120B (en) | Image retrieval method and device based on deep learning | |
| Chen et al. | Image retrieval based on quadtree classified vector quantization | |
| Yakin et al. | Application of content based image retrieval in digital image search system | |
| CN110674334B (en) | Near-repetitive image retrieval method based on consistency region deep learning features | |
| Arica et al. | A perceptual shape descriptor | |
| CN110162654A (en) | It is a kind of that image retrieval algorithm is surveyed based on fusion feature and showing for search result optimization | |
| Polsley et al. | SketchSeeker: finding similar sketches | |
| Zhang et al. | A robust color object analysis approach to efficient image retrieval | |
| Gupta et al. | Comparative study of different low level feature extraction techniques for content based image retrieval | |
| Shambharkar et al. | A comparative study on retrieved images by content based image retrieval system based on binary tree, color, texture and canny edge detection approach | |
| Zou et al. | Sketch-based shape retrieval using pyramid-of-parts |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20211230 Address after: 410008 room 616, building h, tianjianyi square mile, No. 88, Section 1, Furong Middle Road, Kaifu District, Changsha City, Hunan Province Patentee after: Yu Li Address before: 410000 room 1721, building 6, Greenland Central Plaza, Yuelu District, Changsha City, Hunan Province Patentee before: HUNAN SHUNMIAO COMMUNICATION TECHNOLOGY CO.,LTD. |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20220608 Address after: 410205 room 608-52, headquarters building, Changsha CEC Software Park, No. 39, Jianshan Road, high tech Development Zone, Changsha, Hunan Patentee after: Changsha Lansi Intelligent Technology Co.,Ltd. Address before: 410008 room 616, building h, tianjianyi square mile, No. 88, Section 1, Furong Middle Road, Kaifu District, Changsha City, Hunan Province Patentee before: Yu Li |
|
| TR01 | Transfer of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A Hierarchical Image Retrieval Method Based on Convolutional Neural Network Depth Features Effective date of registration: 20231208 Granted publication date: 20210601 Pledgee: Bank of Changsha Limited by Share Ltd. science and Technology Branch Pledgor: Changsha Lansi Intelligent Technology Co.,Ltd. Registration number: Y2023980070454 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PC01 | Cancellation of the registration of the contract for pledge of patent right |
Granted publication date: 20210601 Pledgee: Bank of Changsha Limited by Share Ltd. science and Technology Branch Pledgor: Changsha Lansi Intelligent Technology Co.,Ltd. Registration number: Y2023980070454 |
|
| PC01 | Cancellation of the registration of the contract for pledge of patent right |