CN113326393A - Image retrieval method based on deep hash feature and heterogeneous parallel processing - Google Patents

Image retrieval method based on deep hash feature and heterogeneous parallel processing Download PDF

Info

Publication number
CN113326393A
CN113326393A CN202110600390.6A CN202110600390A CN113326393A CN 113326393 A CN113326393 A CN 113326393A CN 202110600390 A CN202110600390 A CN 202110600390A CN 113326393 A CN113326393 A CN 113326393A
Authority
CN
China
Prior art keywords
hash
image
binary
deep
network model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110600390.6A
Other languages
Chinese (zh)
Other versions
CN113326393B (en
Inventor
廖开阳
陈星�
曹从军
章明珠
王睿天
罗晓洁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Foresight Information Co ltd
Xi'an Huaqi Zhongxin Technology Development Co ltd
Original Assignee
Xian University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian University of Technology filed Critical Xian University of Technology
Priority to CN202110600390.6A priority Critical patent/CN113326393B/en
Publication of CN113326393A publication Critical patent/CN113326393A/en
Application granted granted Critical
Publication of CN113326393B publication Critical patent/CN113326393B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/51—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/55—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Library & Information Science (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Molecular Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses an image retrieval method based on deep hash characteristics and heterogeneous parallel processing, which is implemented according to the following steps: step 1, training a deep Hash network model; step 2, sending the test set and the query image into a trained network model to obtain the depth hash characteristics of the test set and the query image, namely binary hash codes; step 3, calculating Hamming distances between the binary Hash codes of the test set and the query image obtained in the step 2, and sequencing the Hamming distances in an ascending order to obtain a primary ranking result; and 4, selecting binary Hash codes of the previous p images in the initial ranking result, performing Hamming distance calculation again with the binary codes of the query image, and sequencing according to the Hamming distance ascending sequence to obtain a reordering result, namely finally obtaining q retrieval results most similar to the query image. The image retrieval method based on the deep hash feature and heterogeneous parallel processing solves the problem of low image retrieval precision in the prior art.

Description

Image retrieval method based on deep hash feature and heterogeneous parallel processing
Technical Field
The invention belongs to the technical field of computer image retrieval methods, and relates to an image retrieval method based on deep hash characteristics and heterogeneous parallel processing.
Background
With the rapid development of storage devices, computer networks, and multimedia technologies, image data that people have come in contact with and manufactured has increased. Finding out the image that the user wants quickly and accurately in the massive database has become a hot spot of current research, and therefore, the image retrieval technology is also concerned and develops rapidly. There are currently two important challenges with such applications: (1) image features are typically high-dimensional data, storage requirements are high, and computational efficiency is low; (2) the retrieval method of large-scale data has high requirements on speed, time and the like.
In the prior art, two methods are mainly adopted for image retrieval. One is retrieval based on image overall feature description, because the feature dimension is high, all speed influences of storage, operation, retrieval and the like are caused; the other method is to perform retrieval based on local image features, and although the method can accurately describe the local image features, the description of the whole image is lost, so that the retrieval precision is not high.
Therefore, how to provide an image retrieval method to improve the retrieval accuracy and speed is an urgent problem to be solved in the field of computer vision.
Disclosure of Invention
The invention aims to provide an image retrieval method based on deep hash characteristics and heterogeneous parallel processing, and solves the problem of low image retrieval precision in the prior art.
The technical scheme adopted by the invention is that the image retrieval method based on the deep hash feature and heterogeneous parallel processing is implemented according to the following steps:
step 1, off-line training network model
The method comprises the steps of adopting a GoogLeNet network model as an initialization network structure, replacing the last classification layer with a Hash layer, wherein the unit number of the Hash layer is the bit number of an image to be coded, obtaining the GoogLeNet-1 network model, dividing an image data set CIFAR-10 into a training set and a testing set, wherein the training set is divided into 10 types, each type comprises 5000 pieces, and the testing set is divided into 10 types, and each type comprises 1000 pieces.
Inputting the training set into a GoogLeNet-1 network model, extracting image depth features through a convolutional layer, simultaneously performing hash function learning, mapping the depth features through a hash layer to obtain corresponding binary hash codes, and performing iterative optimization and updating on a loss function to obtain optimal network parameters and a final GoogLeNet-hash of the deep hash network model;
step 2, sending the test set and the query image into a trained GoogLeNet-hash network model to obtain the depth hash characteristics of the test set and the query image, namely binary hash codes;
step 3, calculating Hamming distances between the binary Hash codes of the test set and the query image obtained in the step 2, and sequencing the Hamming distances in an ascending order to obtain a primary ranking result;
and 4, selecting binary Hash codes of the previous p images in the initial ranking result, performing Hamming distance calculation again with the binary codes of the query image, and sequencing according to the Hamming distance in an ascending order to obtain a reordering result, namely finally obtaining q retrieval results (q is less than p) which are most similar to the query image.
The present invention is also characterized in that,
the process of generating the binary hash code in the hash layer in the step 1 and the step 2 specifically comprises the following steps:
after an m-dimensional image depth feature x is obtained from a full-connection layer of a GoogleLeNet-hash network model, the x is transmitted to a hash layer, q hash functions are provided on the assumption that the number of nodes of the hash layer is q, q bit hash codes are generated, and the hash codes generated by the q hash functions are shown in the following formula:
(h1,h2,...,hq)T＝(sigmoid(W1x),sigmoid(W2x)...,sigmoid(Wqx))T (1)
wherein h is1-hqFor hash coding of bits 1 to q, sigmoid (W)1x)-sigmoid(Wqx) is the 1 st to q th Hash codes relaxed by sigmoid function, W1-WqTo construct q m-dimensional random vector matrices, W1-Wq∈Rq *m，W1-WqIs generated from a gaussian distribution;
quantizing the relaxed hash code to obtain a final binary hash code H, i.e. H ═ H { (H) }1,h2,...,hq}TThresholding is performed, and the final binary hash code is obtained by the following formula:
Figure BDA0003092503750000031
that is, the binary hash code H is a code consisting of 0 and 1.
In the step 1, iterative optimization and updating are performed on the loss function to obtain the optimal network parameters and the final deep hash network model google lenet-hash, which specifically include:
step 1.1, calculating the probability of each image in the training set belonging to each category;
Figure BDA0003092503750000032
wherein Z iskRepresenting the image features after hash-layer weighting, n representing the number of image classes, f (Z)k) Representing the probability of an image belonging to each class, ZiRepresenting the ith class, wherein 1 < i < n, k is the class of image trueness;
step 1.2, according to f (Z)k) Calculating the value of the Loss function Loss:
Loss＝-logf(Zk) (4)
step 1.3, solving the optimal value of Loss, and updating the weight coefficient theta by adopting a gradient descent method:
Figure BDA0003092503750000033
Figure BDA0003092503750000034
θ＝θ-η(f(Zk)-1+γθ) (7)
wherein gamma is an attenuation factor, and eta is a learning rate, so that correction of the Softmax classifier and updating of network parameters are completed, and a final deep Hash network model GoogLeNet-hash is obtained.
The characteristic extraction in the step 2 is to input the image into a deep hash network GoogLeNet-hash to extract binary hash characteristics of the image and carry out thresholding to finally obtain a characteristic set, and the specific steps are as follows:
i.e. given test set psi ═ I1,I2,...,IgIn which IgRepresenting the g-th image in the test set, inputting the image in the test set into a deep hash network model GoogleLeNet-hash, extracting image hash characteristics and thresholding to obtain a final characteristic set psiH＝{H1,H2,...,HgIn which H isg＝{0,1}q；
Given a query image IkTo query an image IkInputting the image hash characteristics into a deep hash network model GoogLeNet-hash, extracting the image hash characteristics and thresholding the image hash characteristics to obtain a binary hash code H of the imagek；
Wherein HgAnd HkAccording to H ═ H1,h2,...,hq}TAnd then thresholding H according to the formula (3).
The step 3 specifically comprises the following steps:
computing a query image IkBinary hash coding of HkBinary hash coding corresponding to test set imagesSet psiH＝{H1,H2,...,HgH is encoded by each binary hash ofgThe initial search result ordering is obtained according to the distance ascending order of the Hamming distances.
When calculating Hamming distance, the binary Hash code HkAnd binary hash encoding HnComparing each bit, and obtaining the corresponding Hamming distance by comparing whether each bit of the Hash code is the same or not and adding 1 to the Hamming distance if the Hash code is different.
Image I inquiry at CPUkBinary hash coding of HkSet psi of binary hashes corresponding to test set imagesH＝{H1,H2,...,HgAcquisition of H to be acquiredkAnd psiH＝{H1,H2,...,HgAnd transmitting the results to a GPU end of an image processor, calculating the Hamming distance, sequencing the results from small to large according to the Hamming distance after calculation to obtain initial arrangement results, and transmitting the initial arrangement results to a CPU end.
The step 4 specifically comprises the following steps: and the CPU calculates the Hamming distance between the image and the binary Hash codes of the query image again to obtain a reordering result, namely q images q which are most similar to the query image are less than p to obtain a final retrieval result.
60000 CIFAR-10 data sets, 10 training sets in each class and 5000 training sets in each class, and 10 testing sets in each class and 1000 testing sets in each class.
The invention has the beneficial effects that:
the invention combines a deep learning network and a Hash algorithm to form an end-to-end deep Hash network model, then extracts binary Hash codes of CIFAR-10 images as feature indexes, accelerates the retrieval speed by introducing GPU parallel retrieval to carry out feature matching and distance measurement, and finally improves the precision of the final retrieval result by utilizing result rearrangement.
Drawings
FIG. 1 is a flow chart of an image retrieval method based on deep hash feature and heterogeneous parallel processing according to the present invention;
fig. 2 is a schematic diagram of a CPU + GPU heterogeneous parallel processing structure in the image retrieval method based on the deep hash feature and heterogeneous parallel processing of the present invention.
Detailed Description
The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
The invention discloses an image retrieval method based on deep hash characteristics and heterogeneous parallel processing, the flow of which is shown in figure 1 and is specifically implemented according to the following steps:
step 1, off-line training network model
Adopting a GoogLeNet network model as an initialization network structure, replacing the last classification layer with a Hash layer, wherein the unit number of the Hash layer is the bit number of an image to be coded, obtaining a GoogLeNet-1 network model, dividing an image data set CIFAR-10 into a training set and a testing set, wherein the CIFAR-10 data set comprises 60000 data sets, the training set comprises 10 types, each type comprises 5000 data sets, the testing set comprises 10 types and each type comprises 1000 data sets, inputting the training set into the GoogLeNet-1 network model, extracting image depth characteristics through convolution layers, simultaneously performing Hash function learning, mapping the final depth characteristics through the Hash layer to obtain corresponding binary Hash codes, and performing iterative optimization and updating on a loss function to obtain optimal network parameters and a final GoogLeNet-hash of the deep hash network model;
step 2, sending the test set and the query image into a trained GoogLeNet-hash network model to obtain the depth hash characteristics of the test set and the query image, namely binary hash codes;
according to the invention, through designing the hash layer, the parameter values of the hash function are learned from the training data, so as to generate a more compact hash characteristic. After image depth features are obtained from a full connection layer of a GoogLeNet-hash network model, the depth features are transmitted into a hash layer to generate binary hash codes;
the process of generating the binary hash code in the hash layer in the step 1 and the step 2 specifically comprises the following steps:
after an m-dimensional image depth feature x is obtained from a full-connection layer of a GoogleLeNet-hash network model, the x is transmitted to a hash layer, q hash functions are provided on the assumption that the number of nodes of the hash layer is q, q bit hash codes are generated, and the hash codes generated by the q hash functions are shown in the following formula:
(h1,h2,...,hq)T＝(sgn(W1x),sgn(W2x)...,sgn(Wqx))T (1)
since the sgn function is not a convex function and the objective function cannot be optimized and solved by using a gradient-based method, the sigmoid function is selected for relaxation, the coding range is constrained to the (0,1) interval, and the final Hash codes generated by q Hash functions are obtained as shown in the following formula:
(h1,h2,...,hq)T＝(sigmoid(W1x),sigmoid(W2x)...,sigmoid(Wqx))T (2)
wherein h is1-hqFor hash coding of bits 1 to q, sigmoid (W)1x)-sigmoid(Wqx) is the 1 st to q th Hash codes relaxed by sigmoid function, W1-WqTo construct q m-dimensional random vector matrices, W1-Wq∈Rq *m，W1-WqIs generated from a gaussian distribution;
quantizing the relaxed hash code to obtain a final binary hash code H, i.e. H ═ H { (H) }1,h2,...,hq}TThresholding is performed, and the final binary hash code is obtained by the following formula:
Figure BDA0003092503750000071
that is, the binary hash code H is a code consisting of 0 and 1;
the method comprises the following steps of obtaining optimal network parameters and a final GoogLeNet-hash through iterative optimization and updating of a loss function, wherein the step of obtaining the optimal network parameters and the final GoogLeNet-hash is specifically as follows:
step 1.1, calculating the probability of each image in the training set belonging to each category;
Figure BDA0003092503750000072
wherein Z iskRepresenting the image features after hash-layer weighting, n representing the number of image classes, f (Z)k) Representing the probability of an image belonging to each class, ZiRepresenting the ith class, wherein 1 < i < n, k is the class of image trueness;
step 1.2, according to f (Z)k) Calculating the value of the Loss function Loss:
Loss＝-logf(Zk) (5)
step 1.3, solving the optimal value of Loss, and updating the weight coefficient theta by adopting a gradient descent method:
Figure BDA0003092503750000073
Figure BDA0003092503750000074
θ＝θ-η(f(Zk)-1+γθ) (8)
wherein gamma is an attenuation factor, eta is a learning rate, so that correction of the Softmax classifier and updating of network parameters are completed, and a final deep Hash network model GoogLeNet-hash is obtained;
the hash layer also belongs to a hidden layer of a neural network, the number of neurons of the hidden layer is not specifically determined, and the number of nodes of the hash layer designed in the invention determines the length of the binary coding features of the image, so that the number of nodes of the hash layer can be finally determined by comparing the training speed of different node numbers with the precision of the binary coding during retrieval through experiments.
The characteristic extraction in the step 2 is to input the image into a deep hash network GoogLeNet-hash to extract binary hash characteristics of the image and carry out thresholding to finally obtain a characteristic set, and the specific steps are as follows:
i.e. given test set psi ═ I1,I2,...,IgIn which IgRepresenting the g-th image in the test set, inputting the image in the test set into a deep hash network model GoogleLeNet-hash, extracting image hash characteristics and thresholding to obtain a final characteristic set psiH＝{H1,H2,...,HgIn which H isg＝{0,1}q；
Given a query image IkTo query an image IkInputting the image hash characteristics into a deep hash network model GoogLeNet-hash, extracting the image hash characteristics and thresholding the image hash characteristics to obtain a binary hash code H of the imagek；
Wherein HgAnd HkAccording to H ═ H1,h2,...,hq}TAnd then thresholding H according to the formula (3).
Step 3, calculating Hamming distances between the binary Hash codes of the test set and the query image obtained in the step 2, and sequencing the Hamming distances in an ascending order to obtain a primary ranking result; the method specifically comprises the following steps:
the method specifically comprises the following steps:
computing a query image IkBinary hash coding of HkSet psi of binary hashes corresponding to test set imagesH＝{H1,H2,...,HgH is encoded by each binary hash ofgThe initial search result ordering is obtained according to the distance ascending order of the Hamming distances.
When calculating Hamming distance, the binary Hash code HkAnd binary hash encoding HnBy comparing whether each bit of the hash code is the same or not, if different, the hamming distance is increased by 1, e.g., 10001001 and 10110001 have 3 bits different. And if the Hamming distance is 3, obtaining the corresponding Hamming distance, and if the Hamming distance is larger, the difference between the query image and the test set image is larger, namely the similarity is lower. And (4) sorting the Hamming distance from small to large, namely sorting similar images.
As shown in FIG. 2, the image I is inquired at the CPU end of the central processing unitkBinary hash coding of HkSet psi of binary hashes corresponding to test set imagesH＝{H1,H2,...,HgAcquisition of H to be acquiredkAnd psiH＝{H1,H2,...,HgTransmitting the results to a GPU end of an image processor, calculating the Hamming distance, sequencing the results from small to large according to the Hamming distance after calculation to obtain initial arrangement results, and transmitting the initial arrangement results to a CPU end;
and 4, selecting binary hash codes of the previous p images in the initial arrangement result, calculating the Hamming distance between the binary hash codes and the binary hash codes of the query image by the CPU, and sequencing according to the Hamming distance in an ascending order to obtain a reordering result, namely finally obtaining q retrieval results (q is less than p) which are most similar to the query image.
The invention utilizes the deep neural network to extract the characteristics of the image, and the network structure has important influence on the training. Too complicated training difficulty of network structure is big, can appear fitting, and the structure is too simple, can not arouse the learning ability of network. The GoogleLeNet network is selected, the number of layers of the network is increased, loss is increased at different depths to avoid the problem of gradient disappearance, and convolution kernels of different sizes are spliced to achieve the advantage of fusion of features of different scales.
The large-scale image retrieval based on the deep hash feature and heterogeneous parallel processing can be divided into four parts as shown in fig. 1. Respectively as follows: the system comprises a network model training part, an image feature extracting part, a parallel processing and calculating part and a retrieval result reordering part. The training network model part is to replace the last full connection layer of the GoogLeNet to a GoogLeNet-1 network model of a Hash layer, and then obtain a final deep Hash network model GoogLeNet-hash through Hash learning and parameter optimization; the image feature extraction part adopts a pre-trained network model to extract the depth features of the test set image and the query image; the parallel processing and calculating part utilizes the strong data processing capacity of the GPU, divides a thread to calculate the Hamming distance between the query image and the binary Hash codes of the test set image, and carries out similarity sorting according to the distance, wherein the smaller the distance is, the more similar the distance is; the search result rearrangement part is a method for improving the search precision, and obtains a final rearrangement result and the most image q images by calculating the Hamming distance twice.
The invention relates to a large-scale image retrieval method based on deep hash characteristics and heterogeneous parallel processing.A deep hash network model GoogLeNet-hash is obtained through a training set from the aspect of function execution; secondly, extracting binary Hash coding characteristics of the image by adopting a pre-trained deep Hash network model; then, extracting features of the query image, performing feature matching, executing CPU + GPU heterogeneous parallel processing, calculating the Hamming distance of binary Hash codes of the query image and the test set image by a thread, and obtaining an initial sequencing result based on the Hamming distance; and finally, reordering the execution results, and improving the retrieval precision through secondary Hamming distance calculation to obtain q images most similar to the query image. The large-scale image retrieval method based on the depth hash characteristics and heterogeneous parallel processing fully utilizes the depth characteristics of the image and the simplicity of binary hash codes, and combines the strong data processing capacity of the GPU to realize rapid and accurate large-scale image retrieval.

Claims (9)

1. An image retrieval method based on deep hash characteristics and heterogeneous parallel processing is characterized by comprising the following steps:
step 1, off-line training network model
Adopting a GoogLeNet network model as an initialization network structure, replacing the last classification layer with a Hash layer, wherein the unit number of the Hash layer is the bit number of an image to be coded, obtaining a GoogLeNet-1 network model, dividing an image data set CIFAR-10 into a training set and a testing set, wherein the training set and the testing set respectively comprise a plurality of classes of images, inputting the training set into the GoogLeNet-1 network model, extracting image depth characteristics through a convolutional layer, simultaneously performing Hash function learning, mapping the final depth characteristics through the Hash layer to obtain corresponding binary Hash codes, and then performing iterative optimization and updating on a loss function to obtain optimal network parameters and a final GoogLeNet-hash of the deep Hash network model;
step 2, sending the test set and the query image into a trained GoogLeNet-hash network model to obtain the depth hash characteristics of the test set and the query image, namely binary hash codes;
step 3, calculating Hamming distances between the binary Hash codes of the test set and the query image obtained in the step 2, and sequencing the Hamming distances in an ascending order to obtain a primary ranking result;
and 4, selecting binary Hash codes of the previous p images in the initial ranking result, performing Hamming distance calculation again with the binary codes of the query image, and sequencing according to the Hamming distance in an ascending order to obtain a reordering result, namely finally obtaining q retrieval results (q is less than p) which are most similar to the query image.
2. The image retrieval method based on the deep hash feature and the heterogeneous parallel processing according to claim 1, wherein the generating process of generating the binary hash code at the hash layer in the steps 1 and 2 is specifically:
after an m-dimensional image depth feature x is obtained from a full-connection layer of a GoogleLeNet-hash network model, the x is transmitted to a hash layer, q hash functions are provided on the assumption that the number of nodes of the hash layer is q, q bit hash codes are generated, and the hash codes generated by the q hash functions are shown in the following formula:
(h1,h2,...,hq)T＝(sigmoid(W1x),sigmoid(W2x)...,sigmoid(Wqx))T (2)
wherein h is1-hqFor hash coding of bits 1 to q, sigmoid (W)1x)-sigmoid(Wqx) is the 1 st to q th Hash codes relaxed by sigmoid function, W1-WqTo construct q m-dimensional random vector matrices, W1-Wq∈Rq*m，W1-WqIs generated from a gaussian distribution;
quantizing the relaxed hash code to obtain a final binary hash code H, i.e. H ═ H { (H) }1,h2,...,hq}TThresholding is performed, and the final binary hash code is obtained by the following formula:
Figure FDA0003092503740000021
that is, the binary hash code H is a code consisting of 0 and 1.
3. The image retrieval method based on the deep hash feature and the heterogeneous parallel processing as claimed in claim 1, wherein the step 1 further comprises performing iterative optimization and updating on the loss function to obtain the optimal network parameters and the final deep hash network model google net-hash specifically as follows:
step 1.1, calculating the probability of each image in the training set belonging to each category;
Figure FDA0003092503740000022
wherein Z iskRepresenting the image features after hash-layer weighting, n representing the number of image classes, f (Z)k) Representing the probability of an image belonging to each class, ZiRepresenting the ith class, wherein 1 < i < n, k is the class of image trueness;
step 1.2, according to f (Z)k) Calculating the value of the Loss function Loss:
Loss＝-logf(Zk) (5)
step 1.3, solving the optimal value of Loss, and updating the weight coefficient theta by adopting a gradient descent method:
Figure FDA0003092503740000032
Figure FDA0003092503740000031
θ＝θ-η(f(Zk)-1+γθ) (8)
wherein gamma is an attenuation factor, and eta is a learning rate, so that correction of the Softmax classifier and updating of network parameters are completed, and a final deep Hash network model GoogLeNet-hash is obtained.
4. The image retrieval method based on the deep hash feature and the heterogeneous parallel processing as claimed in claim 2, wherein the feature extraction in the step 2 is to input the image into a deep hash network google-hash to extract a binary hash feature of the image and perform thresholding, and finally obtain a feature set, specifically:
i.e. given test set psi ═ I1,I2,...,IgIn which IgRepresenting the g-th image in the test set, inputting the image in the test set into a deep hash network model GoogleLeNet-hash, extracting image hash characteristics and thresholding to obtain a final characteristic set psiH＝{H1,H2,...,HgIn which H isg＝{0,1}q；
Given a query image IkTo query an image IkInputting the image hash characteristics into a deep hash network model GoogLeNet-hash, extracting the image hash characteristics and thresholding the image hash characteristics to obtain a binary hash code H of the imagek；
Wherein HgAnd HkAccording to H ═ H1,h2,...,hq}TAnd then thresholding H according to the formula (3).
5. The image retrieval method based on the deep hash feature and the heterogeneous parallel processing according to claim 4, wherein the step 3 specifically comprises:
computing a query image IkBinary hash coding of HkSet psi of binary hashes corresponding to test set imagesH＝{H1,H2,...,HgH is encoded by each binary hash ofgThe initial search result ordering is obtained according to the distance ascending order of the Hamming distances.
6. According to claim 5The image retrieval method based on the deep hash feature and heterogeneous parallel processing is characterized in that binary hash codes H are coded when Hamming distance is calculatedkAnd binary hash encoding HnComparing each bit, and obtaining the corresponding Hamming distance by comparing whether each bit of the Hash code is the same or not and adding 1 to the Hamming distance if the Hash code is different.
7. The image retrieval method based on the deep hash feature and the heterogeneous parallel processing as claimed in claim 5, wherein the query image I is performed at the CPU end of the central processing unitkBinary hash coding of HkSet psi of binary hashes corresponding to test set imagesH＝{H1,H2,...,HgAcquisition of H to be acquiredkAnd psiH＝{H1,H2,...,HgAnd transmitting the results to a GPU end of an image processor, calculating the Hamming distance, sequencing the results from small to large according to the Hamming distance after calculation to obtain initial arrangement results, and transmitting the initial arrangement results to a CPU end.
8. The image retrieval method based on the deep hash feature and the heterogeneous parallel processing according to claim 7, wherein the step 4 specifically comprises: and the CPU calculates the Hamming distance between the image and the binary Hash codes of the query image again to obtain a reordering result, namely q images q which are most similar to the query image are less than p to obtain a final retrieval result.
9. The image retrieval method based on the deep hash feature and the heterogeneous parallel processing as claimed in claim 1, wherein the CIFAR-10 data sets are 60000 in total, the training sets are 10 in type, each type is 5000 in number, and the testing sets are 10 in type, and each type is 1000 in number.
CN202110600390.6A 2021-05-31 2021-05-31 Image retrieval method based on deep hash feature and heterogeneous parallel processing Active CN113326393B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110600390.6A CN113326393B (en) 2021-05-31 2021-05-31 Image retrieval method based on deep hash feature and heterogeneous parallel processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110600390.6A CN113326393B (en) 2021-05-31 2021-05-31 Image retrieval method based on deep hash feature and heterogeneous parallel processing

Publications (2)

Publication Number Publication Date
CN113326393A true CN113326393A (en) 2021-08-31
CN113326393B CN113326393B (en) 2023-04-07

Family

ID=77422601

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110600390.6A Active CN113326393B (en) 2021-05-31 2021-05-31 Image retrieval method based on deep hash feature and heterogeneous parallel processing

Country Status (1)

Country Link
CN (1) CN113326393B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106407352A (en) * 2016-09-06 2017-02-15 广东顺德中山大学卡内基梅隆大学国际联合研究院 Traffic image retrieval method based on depth learning
CN106503106A (en) * 2016-10-17 2017-03-15 北京工业大学 A kind of image hash index construction method based on deep learning
WO2017092183A1 (en) * 2015-12-03 2017-06-08 中山大学 Image retrieval method based on variable-length deep hash learning
CN107016708A (en) * 2017-03-24 2017-08-04 杭州电子科技大学 A kind of image Hash coding method based on deep learning
CN107423376A (en) * 2017-07-10 2017-12-01 上海交通大学 One kind has the quick picture retrieval method of supervision depth Hash and system
CN108920720A (en) * 2018-07-30 2018-11-30 电子科技大学 The large-scale image search method accelerated based on depth Hash and GPU
CN109241317A (en) * 2018-09-13 2019-01-18 北京工商大学 Based on the pedestrian's Hash search method for measuring loss in deep learning network
CN109241313A (en) * 2018-08-14 2019-01-18 大连大学 A kind of image search method based on the study of high-order depth Hash
CN109918532A (en) * 2019-03-08 2019-06-21 苏州大学 Image search method, device, equipment and computer readable storage medium
US20200073968A1 (en) * 2018-09-04 2020-03-05 Inception Institute of Artificial Intelligence, Ltd. Sketch-based image retrieval techniques using generative domain migration hashing

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017092183A1 (en) * 2015-12-03 2017-06-08 中山大学 Image retrieval method based on variable-length deep hash learning
US20180276528A1 (en) * 2015-12-03 2018-09-27 Sun Yat-Sen University Image Retrieval Method Based on Variable-Length Deep Hash Learning
CN106407352A (en) * 2016-09-06 2017-02-15 广东顺德中山大学卡内基梅隆大学国际联合研究院 Traffic image retrieval method based on depth learning
CN106503106A (en) * 2016-10-17 2017-03-15 北京工业大学 A kind of image hash index construction method based on deep learning
CN107016708A (en) * 2017-03-24 2017-08-04 杭州电子科技大学 A kind of image Hash coding method based on deep learning
CN107423376A (en) * 2017-07-10 2017-12-01 上海交通大学 One kind has the quick picture retrieval method of supervision depth Hash and system
CN108920720A (en) * 2018-07-30 2018-11-30 电子科技大学 The large-scale image search method accelerated based on depth Hash and GPU
CN109241313A (en) * 2018-08-14 2019-01-18 大连大学 A kind of image search method based on the study of high-order depth Hash
US20200073968A1 (en) * 2018-09-04 2020-03-05 Inception Institute of Artificial Intelligence, Ltd. Sketch-based image retrieval techniques using generative domain migration hashing
CN109241317A (en) * 2018-09-13 2019-01-18 北京工商大学 Based on the pedestrian's Hash search method for measuring loss in deep learning network
CN109918532A (en) * 2019-03-08 2019-06-21 苏州大学 Image search method, device, equipment and computer readable storage medium

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JIAN ZHANG等: "SSDH：Semi-supervised Deep Hashing for Large Scale Image Retrieval", 《ARXIV》 *
熊舒羽等: "基于深度哈希的批量图像并行检索方法", 《重庆理工大学学报》 *

Also Published As

Publication number Publication date
CN113326393B (en) 2023-04-07

Similar Documents

Publication Publication Date Title
WO2020182019A1 (en) Image search method, apparatus, device, and computer-readable storage medium
Zhang et al. Improved deep hashing with soft pairwise similarity for multi-label image retrieval
CN111581405B (en) Cross-modal generalization zero sample retrieval method for generating confrontation network based on dual learning
CN108038122B (en) Trademark image retrieval method
CN109783682B (en) Point-to-point similarity-based depth non-relaxed Hash image retrieval method
CN109299342A (en) A kind of cross-module state search method based on circulation production confrontation network
CN110222218B (en) Image retrieval method based on multi-scale NetVLAD and depth hash
CN111125411B (en) Large-scale image retrieval method for deep strong correlation hash learning
KR20210040248A (en) Generative structure-property inverse computational co-design of materials
CN107194378B (en) Face recognition method and device based on mixed dictionary learning
CN110688502A (en) Image retrieval method and storage medium based on depth hash and quantization
CN112732864B (en) Document retrieval method based on dense pseudo query vector representation
CN113806580B (en) Cross-modal hash retrieval method based on hierarchical semantic structure
CN112860930B (en) Text-to-commodity image retrieval method based on hierarchical similarity learning
CN114358188A (en) Feature extraction model processing method, feature extraction model processing device, sample retrieval method, sample retrieval device and computer equipment
CN111008224A (en) Time sequence classification and retrieval method based on deep multitask representation learning
CN114118369A (en) Image classification convolution neural network design method based on group intelligent optimization
CN111026887B (en) Cross-media retrieval method and system
CN108805280B (en) Image retrieval method and device
CN115795065A (en) Multimedia data cross-modal retrieval method and system based on weighted hash code
Liang et al. Cross-media semantic correlation learning based on deep hash network and semantic expansion for social network cross-media search
CN114168782B (en) Deep hash image retrieval method based on triplet network
CN113326393B (en) Image retrieval method based on deep hash feature and heterogeneous parallel processing
CN110135253A (en) A kind of finger vena identification method based on long-term recursive convolution neural network
US11709858B2 (en) Mapping of unlabeled data onto a target schema via semantic type detection

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20230303

Address after: 518000 301, Feiyada Science and Technology Building, No. 002, Gaoxin South 1st Road, High-tech Zone Community, Yuehai Street, Nanshan District, Shenzhen, Guangdong Province

Applicant after: Shenzhen foresight Information Co.,Ltd.

Address before: 710000 No. B49, Xinda Zhongchuang space, 26th Street, block C, No. 2 Trading Plaza, South China City, international port district, Xi'an, Shaanxi Province

Applicant before: Xi'an Huaqi Zhongxin Technology Development Co.,Ltd.

Effective date of registration: 20230303

Address after: 710000 No. B49, Xinda Zhongchuang space, 26th Street, block C, No. 2 Trading Plaza, South China City, international port district, Xi'an, Shaanxi Province

Applicant after: Xi'an Huaqi Zhongxin Technology Development Co.,Ltd.

Address before: 710048 Shaanxi province Xi'an Beilin District Jinhua Road No. 5

Applicant before: XI'AN University OF TECHNOLOGY

GR01 Patent grant
GR01 Patent grant