CN114064948A - Hash image retrieval method and device based on generalized average pooling strategy - Google Patents
Hash image retrieval method and device based on generalized average pooling strategy Download PDFInfo
- Publication number
- CN114064948A CN114064948A CN202111205417.8A CN202111205417A CN114064948A CN 114064948 A CN114064948 A CN 114064948A CN 202111205417 A CN202111205417 A CN 202111205417A CN 114064948 A CN114064948 A CN 114064948A
- Authority
- CN
- China
- Prior art keywords
- image
- feature
- hash
- retrieved
- feature extraction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011176 pooling Methods 0.000 title claims abstract description 53
- 238000000034 method Methods 0.000 title claims abstract description 50
- 239000013598 vector Substances 0.000 claims abstract description 93
- 238000000605 extraction Methods 0.000 claims abstract description 81
- 238000013507 mapping Methods 0.000 claims abstract description 31
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 25
- 230000008569 process Effects 0.000 claims abstract description 14
- 238000012549 training Methods 0.000 claims description 20
- 230000004927 fusion Effects 0.000 claims description 13
- 238000007781 pre-processing Methods 0.000 claims description 13
- 238000012545 processing Methods 0.000 claims description 8
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 7
- 239000011159 matrix material Substances 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000019771 cognition Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
- G06F16/532—Query formulation, e.g. graphical querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Abstract
The invention relates to a Hash image retrieval method and a Hash image retrieval device based on a generalized average pooling strategy, wherein the method comprises the following steps: establishing a feature index of an image feature library; performing feature extraction on the image to be retrieved by using a pre-trained feature extraction network to obtain a feature vector of the image to be retrieved; mapping the characteristic vectors of the image to be retrieved by using a locality sensitive hashing algorithm, and mapping the characteristic vectors into a hashing bucket; searching and obtaining an image feature vector corresponding to the hash bucket according to the feature index; and calculating the similarity between the image feature vector to be retrieved and the image feature vector corresponding to the hash bucket, sorting according to the distance from small to large, outputting the first k retrieval results, and finishing the image retrieval when k is a positive integer. The retrieval method can accelerate the process of retrieving the similarity images in the image library and realize the quick matching of the images to be retrieved in the image library.
Description
Technical Field
The invention belongs to the technical field of digital image processing, and particularly relates to a Hash image retrieval method and a Hash image retrieval device based on a generalized average pooling strategy.
Background
The image retrieval technology aims at image contents which are interested by a user, and presents related images to the user in a way that the similarity is from top to bottom according to a specific similarity measurement standard. The core problem is how to condense the information of the image, obtain the feature descriptor of the image and fully express the content information of the image. In recent years, with the rapid development of internet technology, a great deal of information is overloaded, and how to conveniently, rapidly and accurately query and retrieve image data in a huge image library is still a problem to be solved.
The large-scale image retrieval problem has the problems of high dimensionality, large data size, time consumption in calculation and the like. The retrieval problem can be viewed as a nearest neighbor search problem in nature. In the traditional searching mode such as a classical kd-tree, an R-tree and the like, under the background of mass data, the time and space overhead which is difficult to imagine is required for obtaining an accurate nearest neighbor searching result.
In an image retrieval system, effective extraction of semantic features of images is very critical. In recent years, with the popularization of deep learning technology, a large number of feature extraction algorithms gradually start to perform feature extraction by using deep learning, and pooling operation is to process each feature based on feature extraction and then fuse the obtained data to obtain a final result. The traditional pooling strategy treats the extracted local feature map equally, is not in accordance with the nonlinear characteristics of a human visual system, and the problems of missing effective feature information of the image, low retrieval speed and the like exist in the existing retrieval method under large-scale data.
Disclosure of Invention
In order to solve the above problems in the prior art, the present invention provides a hash image retrieval method and apparatus based on a generalized average pooling policy. The technical problem to be solved by the invention is realized by the following technical scheme:
the invention provides a Hash image retrieval method based on a generalized average pooling strategy, which comprises the following steps:
establishing a feature index of an image feature library;
performing feature extraction on the image to be retrieved by using a pre-trained feature extraction network to obtain a feature vector of the image to be retrieved;
mapping the characteristic vector of the image to be retrieved by using a locality sensitive hashing algorithm, and mapping the characteristic vector of the image to be retrieved into a hashing bucket;
searching and obtaining an image feature vector corresponding to the hash bucket according to the feature index;
and calculating the similarity between the image feature vector to be retrieved and the image feature vector corresponding to the hash bucket, sorting according to the distance from small to large, outputting the first k retrieval results, and finishing the image retrieval when the k value is a positive integer.
In one embodiment of the present invention, establishing a feature index of an image feature library comprises:
performing feature extraction on images in a preset image data set by using a pre-trained feature extraction network to construct and obtain an image feature library;
mapping image feature vectors in the image feature library by using a locality sensitive hashing algorithm, mapping the image feature vectors to hash buckets, and establishing indexes between the hash buckets and the image feature vectors as feature indexes;
wherein similar image feature vectors are mapped to the same hash bucket.
In one embodiment of the invention, the feature extraction network comprises a cascade of a feature extraction unit and a feature fusion unit, wherein,
the feature extraction unit is a ResNet101 convolutional neural network and is used for extracting features of an input image to obtain a plurality of feature maps;
the feature fusion unit is a generalized average pooling layer and is used for performing feature fusion on the feature maps to obtain image feature vectors.
In an embodiment of the present invention, the feature extraction is performed on an image to be retrieved by using a pre-trained feature extraction network, and the method further includes:
and preprocessing the image to be retrieved, and scaling the size of the image to the image size preset by the feature extraction network.
In an embodiment of the present invention, the pre-trained feature extraction network is obtained by training through the following steps:
step a: acquiring a training data set;
step b: pre-processing each image in the training data set to scale its size to 1024 x 1024;
step c: inputting each preprocessed image into a preset feature extraction network so that the feature extraction network performs feature extraction and feature fusion on the preprocessed image to obtain a corresponding image feature vector;
step d: and d, adjusting the network parameters of the preset feature extraction network, and repeating the steps c to d until a preset training cut-off condition is reached to obtain the trained feature extraction network.
In an embodiment of the present invention, the feature extraction network further includes a data enhancement unit configured to perform data enhancement processing on the input image.
In an embodiment of the present invention, the feature extraction network further includes a regularization unit for preventing the network from being over-fitted during the training process.
The invention provides a Hash image retrieval device based on a generalized average pooling strategy, which comprises the following steps:
the acquisition module is used for acquiring an image to be retrieved;
the image feature extraction module is used for extracting features of the image to be retrieved according to the pre-trained feature extraction network to obtain a feature vector of the image to be retrieved;
the mapping module is used for mapping the image feature vector to be retrieved according to a locality sensitive hashing algorithm and mapping the image feature vector to be retrieved into a hashing bucket;
the index module is used for searching and obtaining an image characteristic vector corresponding to the hash bucket according to a characteristic index of a pre-established image characteristic library;
and the retrieval module is used for calculating the similarity between the image feature vector to be retrieved and the image feature vector corresponding to the hash bucket, sorting the image feature vectors according to the distance from small to large, outputting the first k retrieval results, and finishing the image retrieval when the k value is a positive integer.
In an embodiment of the present invention, the hash image retrieval apparatus based on the generalized average pooling policy further includes:
and the preprocessing module is used for preprocessing the image to be retrieved and scaling the size of the image to the image size preset by the feature extraction network.
In an embodiment of the present invention, the hash image retrieval apparatus based on the generalized average pooling policy further includes: the feature index establishing module is specifically configured to:
extracting the features of the images in a preset image data set according to the feature extraction network to construct and obtain an image feature library;
and mapping the image feature vectors in the image feature library by using a locality sensitive hashing algorithm, mapping the image feature vectors to a hash bucket, and establishing an index between the hash bucket and the image feature vectors as the feature index.
Compared with the prior art, the invention has the beneficial effects that:
1. the Hash image retrieval method based on the generalized average pooling strategy adopts the generalized average pooling strategy in the characteristic extraction process, and can effectively simulate the cognitive process of a human visual system on an image, so that abundant characteristic information in the image is effectively extracted;
2. the Hash image retrieval method based on the generalized average pooling strategy maps the characteristic information into a Hash bucket through a locality sensitive Hash algorithm in the retrieval process, utilizes the characteristic index of a pre-established image characteristic library to query the image characteristic vector corresponding to the Hash bucket, and calculates the similarity between the image characteristic vector and the characteristic information to obtain the retrieval result, thereby quickening the process of retrieving the similar image in the image library and realizing the quick matching of the image to be retrieved in the image library.
The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical means of the present invention more clearly understood, the present invention may be implemented in accordance with the content of the description, and in order to make the above and other objects, features, and advantages of the present invention more clearly understood, the following preferred embodiments are described in detail with reference to the accompanying drawings.
Drawings
Fig. 1 is a flowchart of a hash image retrieval method based on a generalized average pooling policy according to an embodiment of the present invention;
fig. 2 is a schematic flowchart of a hash image retrieval method based on a generalized average pooling policy according to an embodiment of the present invention;
fig. 3 is a block diagram of a feature extraction network according to an embodiment of the present invention;
fig. 4 is a block diagram of a hash image retrieval apparatus based on a generalized average pooling policy according to an embodiment of the present invention.
Detailed Description
To further illustrate the technical means and effects of the present invention adopted to achieve the predetermined object, the following describes in detail a hash image retrieval method and apparatus based on the generalized average pooling scheme according to the present invention with reference to the accompanying drawings and the detailed description.
The foregoing and other technical matters, features and effects of the present invention will be apparent from the following detailed description of the embodiments, which is to be read in connection with the accompanying drawings. The technical means and effects of the present invention adopted to achieve the predetermined purpose can be more deeply and specifically understood through the description of the specific embodiments, however, the attached drawings are provided for reference and description only and are not used for limiting the technical scheme of the present invention.
Example one
Referring to fig. 1 and fig. 2 in combination, fig. 1 is a flowchart of a hash image retrieval method based on a generalized average pooling policy according to an embodiment of the present invention, and fig. 2 is a flowchart of the hash image retrieval method based on the generalized average pooling policy according to the embodiment of the present invention. As shown in the figure, the hash image retrieval method based on the generalized average pooling policy of the embodiment includes:
s1: establishing a feature index of an image feature library;
s2: performing feature extraction on the image to be retrieved by using a pre-trained feature extraction network to obtain a feature vector of the image to be retrieved;
s3: mapping the characteristic vectors of the image to be retrieved by using a locality sensitive hashing algorithm, and mapping the characteristic vectors into a hashing bucket;
s4: searching and obtaining an image feature vector corresponding to the hash bucket according to the feature index;
s5: and calculating the similarity between the image feature vector to be retrieved and the image feature vector corresponding to the hash bucket, sorting according to the distance from small to large, outputting the first k retrieval results, and finishing the image retrieval when k is a positive integer.
Further, step S1 is preceded by:
s0: and preprocessing the image to be retrieved, and scaling the size of the image to the image size preset by the feature extraction network.
In this embodiment, the input resolution of the feature extraction network is 1024 × 1024.
Specifically, step S1 includes:
s11: performing feature extraction on images in a preset image data set by using a pre-trained feature extraction network to construct an image feature library;
it should be noted that, before the images in the image data set are input into the pre-trained feature extraction network, the images also need to be preprocessed, and the size of the images is scaled to 1024 × 1024.
S12: and mapping the image feature vectors in the image feature library by using a locality sensitive hashing algorithm, mapping the image feature vectors to a hash bucket, and establishing an index between the hash bucket and the image feature vectors as a feature index.
Wherein similar image feature vectors are mapped to the same hash bucket.
Referring to fig. 3, fig. 3 is a block diagram of a feature extraction network according to an embodiment of the present invention. As shown in the figure, the feature extraction network of the present embodiment includes a feature extraction unit and a feature fusion unit, which are cascaded, where the feature extraction unit is configured to perform feature extraction on an input image to obtain a plurality of feature maps. The feature fusion unit is used for performing feature fusion on the feature maps to obtain image feature vectors.
In this embodiment, the feature extraction unit is a ResNet101 convolutional neural network, and the feature fusion unit is a generalized average pooling layer.
Further, the feature extraction network of the embodiment further includes a data enhancement unit and a regularization unit, so as to avoid the problem of over-fitting or under-fitting of the network in the training process, and ensure model convergence by adjusting the learning rate in the network training process.
The data enhancement unit is used for performing data enhancement processing on the input image, and the data enhancement processing comprises the processing of turning, rotating, scaling, cutting or shifting the input image.
It should be noted that, in this embodiment, data enhancement may be performed on the input image by adding a dropout layer in the network.
In this embodiment, the generalized average pooling operation is performed on the entire feature map, and when an image (1024 × 1024) of a specific size passes through the ResNet101 convolutional neural network, a size W is generated1*H1K, where K represents the number of channels of the output signature. Through the above process, the generated 3-dimensional tensor can be regarded as a set of 2-dimensional feature maps, which can be expressed mathematically as: c ═ Ci1, i ═ 1 … K, where CiDenoted as the 2-dimensional feature map of the ith channel.
In general, CiAre averaged or maximum pooled into a one-dimensional vector f, which is then represented as a feature vector for the image. However, the above average pooling tends to ignore the importance of local information in the forward or backward propagation process; maximum pooling only retains the response points with the largest response values during forward or backward propagation, which makes the feature certain discriminant, but lacks itThe correlation between the convolution responses is lacking.
To solve this problem, inspired by the generalized mean, the average pooling and the maximum pooling, as a special case of the generalized average pooling, can be expressed as:
as can be seen from the above formula, when α tends to infinity, it is the maximum pooling operation, and when α is 1, it is the average pooling operation. Alpha is obtained through training as a network training parameter, so that the features obtained through generalized average pooling retain the discriminant of the maximum value pooling feature and the correlation of the average pooling, and a more effective feature vector is obtained.
The hash image retrieval method based on the generalized average pooling strategy of the embodiment adopts the generalized average pooling strategy in the feature extraction process, and can effectively simulate the cognitive process of a human visual system on an image, thereby effectively extracting rich characteristic information in the image.
Further, the feature extraction network trained in advance in this embodiment is obtained by training through the following steps:
step a: acquiring a training data set;
step b: preprocessing each image in the training dataset to scale its size to 1024 x 1024;
step c: inputting each preprocessed image into a preset feature extraction network so that the feature extraction network performs feature extraction and feature fusion on the preprocessed image to obtain an image feature vector corresponding to the preprocessed image feature vector;
step d: and (d) adjusting the network parameters of the preset feature extraction network, and repeating the steps c to d until a preset training cut-off condition is reached to obtain the trained feature extraction network.
In this embodiment, the training cutoff condition is that the total loss function of the feature extraction network is minimal or reaches the training times.
Further, the hash algorithm is also called as a hash algorithm, and the conventional hash algorithm refers to constructing a hash model, and using the hash model, the high-dimensional picture features can be mapped into low-dimensional hash codes, that is, the text can be reduced into simple numeric strings through the hash model, and then whether two images are similar or not is determined by calculating and comparing the similarity between two numeric strings.
The locality sensitive hashing algorithm is generally used after the conventional hashing, and compared with pairwise comparison, the locality sensitive hashing algorithm can achieve dimension reduction and locality search for matching pairs. In this embodiment, the specific process of the locality sensitive hashing algorithm includes:
firstly, forming a matrix by image feature vectors in an image feature library;
secondly, for the first hash, hashing a Matrix formed by image feature vectors into a ' Signature Matrix ' (Signature Matrix ') by using hash functions (including a plurality of hash functions) selected from any hash function family, wherein the Signature Matrix can be directly understood as data after dimensionality reduction, and simhash and minhash can be used for carrying out hash operation in the step, and hash operation can also be carried out by using different functions;
and finally, the second time of hashing is to perform hashing operation on the Signature Matrix to obtain a final hashing result, namely, the hash bucket to which each image feature vector is hashed finally.
It should be noted that, in the locality sensitive hashing algorithm, the more similar data (image feature vectors) in the original space have higher probability of falling into the same hash bucket after passing through the same mapping function. Therefore, the locality sensitive hashing algorithm directly uses the randomly generated parameters as the hashing function, and can achieve high accuracy.
Specifically, the retrieval process implemented in this embodiment is specifically described, in this embodiment, for an image to be retrieved, an image feature vector of the image to be retrieved is obtained through feature extraction network extraction, and is used as an image feature vector to be retrieved, the image feature vector to be retrieved is used as a new data point, a local sensitive hashing algorithm is used to perform a hashing operation on the image feature vector to be retrieved, so as to obtain a final hash result of the image feature vector to be retrieved, that is, to which hash bucket the image feature vector to be retrieved is hashed finally, then an index (that is, a feature index) between the established hash bucket and an image feature vector in an image feature library is used to index an image feature vector in the image feature library corresponding to the hash bucket, and finally, the image feature vector in the image feature library corresponding to the hash bucket is indexed, and carrying out pairwise calculation to compare the similarity, sorting according to the distance from small to large, outputting the first k retrieval results, and finishing the retrieval when k is a positive integer.
In the hash image retrieval method based on the generalized average pooling strategy, the feature information is mapped into the hash bucket through a locality sensitive hash algorithm in the retrieval process, the image feature vector corresponding to the hash bucket is inquired by using the feature index pre-established in the image feature library, the similarity between the image feature vector and the feature information is calculated, the retrieval result is obtained, the process of retrieving the similar image in the image library is accelerated, and the quick matching of the image to be retrieved in the image library is realized.
Example two
Fig. 4 shows a block diagram of a hash image retrieval apparatus based on a generalized average pooling policy according to an embodiment of the present invention, where fig. 4 is a block diagram of a hash image retrieval apparatus based on a generalized average pooling policy according to an embodiment of the present invention. As shown in the figure, the hash image retrieval apparatus based on the generalized average pooling policy of the present embodiment includes: the device comprises an acquisition module, an image feature extraction module, a mapping module, an index module and a retrieval module. The acquisition module is used for acquiring an image to be retrieved; the image feature extraction module is used for extracting features of the image to be retrieved according to the pre-trained feature extraction network to obtain a feature vector of the image to be retrieved; the mapping module is used for mapping the characteristic vector of the image to be retrieved according to the locality sensitive hash algorithm and mapping the characteristic vector into a hash bucket; the index module is used for searching and obtaining an image characteristic vector corresponding to the hash bucket according to a characteristic index of a pre-established image characteristic library; the retrieval module is used for calculating the similarity between the image feature vector to be retrieved and the image feature vector corresponding to the hash bucket, sorting the image feature vectors according to the distance from small to large, outputting the first k retrieval results, and finishing the image retrieval when k is a positive integer.
Further, the hash image retrieval device based on the generalized average pooling policy of the embodiment further includes a preprocessing module and a feature index establishing module. The pre-processing module is used for pre-processing the image to be retrieved and scaling the size of the image to the image size preset by the feature extraction network. In this embodiment, the input resolution of the feature extraction network is 1024 × 1024. The characteristic index establishing module is specifically used for extracting the characteristics of the images in the preset image data set according to the characteristic extracting network and establishing an image characteristic library; and mapping the image feature vectors in the image feature library by using a locality sensitive hashing algorithm, mapping the image feature vectors to a hash bucket, and establishing an index between the hash bucket and the image feature vectors as a feature index.
Based on the same inventive concept, the embodiment of the invention also provides electronic equipment, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus. The memory is used for storing computer programs; the processor is configured to implement the method steps of any one of the above hash image retrieval methods based on the generalized average pooling policy when executing the program stored in the memory.
In practical applications, the electronic device may be: monitoring devices, image processing devices, desktop computers, portable computers, intelligent mobile terminals, and the like. Without limitation, any electronic device that can implement the present invention is within the scope of the present invention.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
The invention also provides a computer readable storage medium. In this computer readable storage medium a computer program is stored which, when being executed by a processor, carries out the method steps of any of the above-mentioned hash image retrieval methods based on a generalized average pooling policy.
Alternatively, the computer-readable storage medium may be a Non-Volatile Memory (NVM), such as at least one disk Memory. Alternatively, the computer readable storage medium may be at least one memory device located remotely from the processor.
In a further embodiment of the present invention, there is also provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method steps of any of the above-described hash image retrieval methods based on a generalized average pooling policy.
For the electronic device/storage medium/computer program product embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, refer to the partial description of the method embodiment.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that an article or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of additional like elements in the article or device comprising the element. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.
Claims (10)
1. A Hash image retrieval method based on a generalized average pooling strategy is characterized by comprising the following steps:
establishing a feature index of an image feature library;
performing feature extraction on the image to be retrieved by using a pre-trained feature extraction network to obtain a feature vector of the image to be retrieved;
mapping the characteristic vector of the image to be retrieved by using a locality sensitive hashing algorithm, and mapping the characteristic vector of the image to be retrieved into a hashing bucket;
searching and obtaining an image feature vector corresponding to the hash bucket according to the feature index;
and calculating the similarity between the image feature vector to be retrieved and the image feature vector corresponding to the hash bucket, sorting according to the distance from small to large, outputting the first k retrieval results, and finishing the image retrieval when the k value is a positive integer.
2. The hash image retrieval method based on the generalized average pooling strategy of claim 1, wherein establishing a feature index of an image feature library comprises:
performing feature extraction on images in a preset image data set by using a pre-trained feature extraction network to construct and obtain an image feature library;
mapping image feature vectors in the image feature library by using a locality sensitive hashing algorithm, mapping the image feature vectors to hash buckets, and establishing indexes between the hash buckets and the image feature vectors as feature indexes;
wherein similar image feature vectors are mapped to the same hash bucket.
3. The hash image retrieval method based on the generalized average pooling strategy of claim 1, wherein said feature extraction network comprises a cascade of a feature extraction unit and a feature fusion unit, wherein,
the feature extraction unit is a ResNet101 convolutional neural network and is used for extracting features of an input image to obtain a plurality of feature maps;
the feature fusion unit is a generalized average pooling layer and is used for performing feature fusion on the feature maps to obtain image feature vectors.
4. The hash image retrieval method based on the generalized average pooling strategy of claim 1, wherein the feature extraction is performed on the image to be retrieved by using a pre-trained feature extraction network, and the method further comprises:
and preprocessing the image to be retrieved, and scaling the size of the image to the image size preset by the feature extraction network.
5. The hash image retrieval method based on the generalized average pooling strategy of claim 1, wherein the pre-trained feature extraction network is obtained by training through the following steps:
step a: acquiring a training data set;
step b: pre-processing each image in the training data set to scale its size to 1024 x 1024;
step c: inputting each preprocessed image into a preset feature extraction network so that the feature extraction network performs feature extraction and feature fusion on the preprocessed image to obtain a corresponding image feature vector;
step d: and d, adjusting the network parameters of the preset feature extraction network, and repeating the steps c to d until a preset training cut-off condition is reached to obtain the trained feature extraction network.
6. The method as claimed in claim 5, wherein the feature extraction network further comprises a data enhancement unit for performing data enhancement processing on the input image.
7. The method as claimed in claim 5, wherein the feature extraction network further comprises a regularization unit for preventing the network from being over-fitted during the training process.
8. A hash image retrieval apparatus based on a generalized average pooling policy, comprising:
the acquisition module is used for acquiring an image to be retrieved;
the image feature extraction module is used for extracting features of the image to be retrieved according to the pre-trained feature extraction network to obtain a feature vector of the image to be retrieved;
the mapping module is used for mapping the image feature vector to be retrieved according to a locality sensitive hashing algorithm and mapping the image feature vector to be retrieved into a hashing bucket;
the index module is used for searching and obtaining an image characteristic vector corresponding to the hash bucket according to a characteristic index of a pre-established image characteristic library;
and the retrieval module is used for calculating the similarity between the image feature vector to be retrieved and the image feature vector corresponding to the hash bucket, sorting the image feature vectors according to the distance from small to large, outputting the first k retrieval results, and finishing the image retrieval when the k value is a positive integer.
9. The hash image retrieval apparatus based on the generalized average pooling strategy of claim 8, further comprising:
and the preprocessing module is used for preprocessing the image to be retrieved and scaling the size of the image to the image size preset by the feature extraction network.
10. The hash image retrieval apparatus based on the generalized average pooling strategy of claim 8, further comprising: the feature index establishing module is specifically configured to:
extracting the features of the images in a preset image data set according to the feature extraction network to construct and obtain an image feature library;
and mapping the image feature vectors in the image feature library by using a locality sensitive hashing algorithm, mapping the image feature vectors to a hash bucket, and establishing an index between the hash bucket and the image feature vectors as the feature index.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111205417.8A CN114064948A (en) | 2021-10-15 | 2021-10-15 | Hash image retrieval method and device based on generalized average pooling strategy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111205417.8A CN114064948A (en) | 2021-10-15 | 2021-10-15 | Hash image retrieval method and device based on generalized average pooling strategy |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114064948A true CN114064948A (en) | 2022-02-18 |
Family
ID=80234694
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111205417.8A Pending CN114064948A (en) | 2021-10-15 | 2021-10-15 | Hash image retrieval method and device based on generalized average pooling strategy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114064948A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115115523A (en) * | 2022-08-26 | 2022-09-27 | 中加健康工程研究院(合肥)有限公司 | CNN and Transformer fused medical image depth information extraction method |
| CN115733617A (en) * | 2022-10-31 | 2023-03-03 | 支付宝(杭州)信息技术有限公司 | Biological characteristic authentication method and system |
| CN116680434A (en) * | 2023-07-28 | 2023-09-01 | 腾讯科技（深圳）有限公司 | Image retrieval method, device, equipment and storage medium based on artificial intelligence |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101404032A (en) * | 2008-11-11 | 2009-04-08 | 清华大学 | Video retrieval method and system based on contents |
| CN105302833A (en) * | 2014-07-24 | 2016-02-03 | 上海极誉实业有限公司 | Content based video retrieval mathematic model establishment method |
| CN106202362A (en) * | 2016-07-07 | 2016-12-07 | Tcl集团股份有限公司 | Image recommendation method and image recommendation device |
| CN107180079A (en) * | 2017-04-28 | 2017-09-19 | 华南理工大学 | The image search method of index is combined with Hash based on convolutional neural networks and tree |
| CN107577990A (en) * | 2017-08-09 | 2018-01-12 | 武汉世纪金桥安全技术有限公司 | A kind of extensive face identification method for accelerating retrieval based on GPU |
| CN110825904A (en) * | 2019-10-24 | 2020-02-21 | 腾讯科技（深圳）有限公司 | Image matching method and device, electronic equipment and storage medium |
| CN113157961A (en) * | 2021-03-11 | 2021-07-23 | 深圳大学 | Method for generating combined non-category area suggestion box and quickly retrieving image target |
-
2021
- 2021-10-15 CN CN202111205417.8A patent/CN114064948A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101404032A (en) * | 2008-11-11 | 2009-04-08 | 清华大学 | Video retrieval method and system based on contents |
| CN105302833A (en) * | 2014-07-24 | 2016-02-03 | 上海极誉实业有限公司 | Content based video retrieval mathematic model establishment method |
| CN106202362A (en) * | 2016-07-07 | 2016-12-07 | Tcl集团股份有限公司 | Image recommendation method and image recommendation device |
| CN107180079A (en) * | 2017-04-28 | 2017-09-19 | 华南理工大学 | The image search method of index is combined with Hash based on convolutional neural networks and tree |
| CN107577990A (en) * | 2017-08-09 | 2018-01-12 | 武汉世纪金桥安全技术有限公司 | A kind of extensive face identification method for accelerating retrieval based on GPU |
| CN110825904A (en) * | 2019-10-24 | 2020-02-21 | 腾讯科技（深圳）有限公司 | Image matching method and device, electronic equipment and storage medium |
| CN113157961A (en) * | 2021-03-11 | 2021-07-23 | 深圳大学 | Method for generating combined non-category area suggestion box and quickly retrieving image target |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115115523A (en) * | 2022-08-26 | 2022-09-27 | 中加健康工程研究院(合肥)有限公司 | CNN and Transformer fused medical image depth information extraction method |
| CN115733617A (en) * | 2022-10-31 | 2023-03-03 | 支付宝(杭州)信息技术有限公司 | Biological characteristic authentication method and system |
| CN115733617B (en) * | 2022-10-31 | 2024-01-23 | 支付宝(杭州)信息技术有限公司 | Biological feature authentication method and system |
| CN116680434A (en) * | 2023-07-28 | 2023-09-01 | 腾讯科技（深圳）有限公司 | Image retrieval method, device, equipment and storage medium based on artificial intelligence |
| CN116680434B (en) * | 2023-07-28 | 2023-11-17 | 腾讯科技（深圳）有限公司 | Image retrieval method, device, equipment and storage medium based on artificial intelligence |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110162593B (en) | Search result processing and similarity model training method and device | |
| CN110866140B (en) | Image feature extraction model training method, image searching method and computer equipment | |
| CN106682233B (en) | Hash image retrieval method based on deep learning and local feature fusion | |
| CN107133213B (en) | Method and system for automatically extracting text abstract based on algorithm | |
| US20190108242A1 (en) | Search method and processing device | |
| CN114064948A (en) | Hash image retrieval method and device based on generalized average pooling strategy | |
| CN110222218B (en) | Image retrieval method based on multi-scale NetVLAD and depth hash | |
| CN105975459B (en) | A kind of the weight mask method and device of lexical item | |
| Varish et al. | Content based image retrieval using statistical features of color histogram | |
| WO2020114100A1 (en) | Information processing method and apparatus, and computer storage medium | |
| WO2023029356A1 (en) | Sentence embedding generation method and apparatus based on sentence embedding model, and computer device | |
| CN111274445B (en) | Similar video content retrieval method and system based on triple deep learning | |
| CN111159485A (en) | Tail entity linking method, device, server and storage medium | |
| Li et al. | Hashing with dual complementary projection learning for fast image retrieval | |
| CN115203421A (en) | Method, device and equipment for generating label of long text and storage medium | |
| CN105320764A (en) | 3D model retrieval method and 3D model retrieval apparatus based on slow increment features | |
| CN113691542A (en) | Web attack detection method based on HTTP request text and related equipment | |
| CN112632261A (en) | Intelligent question and answer method, device, equipment and storage medium | |
| CN113806580B (en) | Cross-modal hash retrieval method based on hierarchical semantic structure | |
| US11914641B2 (en) | Text to color palette generator | |
| Sun et al. | Search by detection: Object-level feature for image retrieval | |
| US20200364259A1 (en) | Image retrieval | |
| CN113722512A (en) | Text retrieval method, device and equipment based on language model and storage medium | |
| Zheng et al. | Deep learning hash for wireless multimedia image content security | |
| Yuan et al. | OSAP‐Loss: Efficient optimization of average precision via involving samples after positive ones towards remote sensing image retrieval |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220218 |
|
| RJ01 | Rejection of invention patent application after publication |