CN111177435A - CBIR method based on improved PQ algorithm - Google Patents

CBIR method based on improved PQ algorithm Download PDF

Info

Publication number
CN111177435A
CN111177435A CN201911417377.6A CN201911417377A CN111177435A CN 111177435 A CN111177435 A CN 111177435A CN 201911417377 A CN201911417377 A CN 201911417377A CN 111177435 A CN111177435 A CN 111177435A
Authority
CN
China
Prior art keywords
vector
sample
retrieval
query
index
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911417377.6A
Other languages
Chinese (zh)
Other versions
CN111177435B (en
Inventor
曾浩
高凡
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University of Post and Telecommunications
Original Assignee
Chongqing University of Post and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University of Post and Telecommunications filed Critical Chongqing University of Post and Telecommunications
Priority to CN201911417377.6A priority Critical patent/CN111177435B/en
Publication of CN111177435A publication Critical patent/CN111177435A/en
Application granted granted Critical
Publication of CN111177435B publication Critical patent/CN111177435B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/51—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/53—Querying
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/55—Clustering; Classification
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to a CBIR method based on an improved PQ algorithm, belonging to the technical field of image processing. The method comprises the steps of extracting image depth features through an improved depth convolution network, coding and compressing image feature data through an index retrieval module of an inverted index based product quantification IVPQ algorithm which adopts a nonlinear retrieval ANN search strategy, generating indexes of a dynamic index database based on a Faiss frame, partitioning a data space of a full index database through feature vector coding, rapidly locking a certain subspace through Hamming distance rearrangement for traversal when retrieval of a query picture is carried out, and outputting a retrieval image. The invention realizes the dynamism of the retrieval index library based on the Faiss framework, and avoids the high operation and maintenance cost generated for reconstructing the index library in practical application occasions.

Description

CBIR method based on improved PQ algorithm
Technical Field
The invention belongs to the field of image processing, and relates to a CBIR method based on an improved PQ algorithm.
Background
In an actual application scene, a user needs to search and judge a key-sensitive image library based on massive, label-free and complex unknown images, so that the function of searching images with images is realized. At present, the most effective mode for representing the index image information is based on the image Content, so a Content-based image Retrieval (CBIR) method is selected for large-scale image Retrieval system design.
The traditional CBIR method employs a Brute-force (Brute-force) strategy for similarity measurement, which aggravates the consumption of memory resources as the picture characteristic index data increases. Particularly, when the data set scale of the actual application occasion reaches hundreds of millions of scales, the occupation of the operating memory (RAM) cannot be met due to the increase of the index scale, so that the retrieval performance is sharply reduced, the system performance cannot obtain the expected target, and the hardware cost is sharply increased. For this reason, the mainstream solution is to use an Approximate Nearest Neighbor (ANN) search strategy, which essentially partitions the full space of the search data set into subspaces, and locks (several) subspace sets in a certain way and traverses them quickly.
The ANN is mainly classified into a KD tree method, a graph index quantization method, a hash method, and a vector quantization method. For a conventional KD tree algorithm, along with the deeper tree depth of a KD tree, the performance of a KD tree method retrieval model is poor; the method for introducing the graph into the ANN search takes a mature HNSW (Hierarchical navigation Small World graphics) algorithm as an example, the recall rate is high, the index memory occupies a large amount, and a unique index structure is not beneficial to dynamic addition and deletion of data; for the hash method, a multi-table Locality-Sensitive Hashing (MLSH) is generated, which is an improved hash coding algorithm for generating a plurality of hash functions for dividing a space domain by constructing a plurality of hash tables so as to improve the search accuracy under a large-scale high-dimensional data set, but the condition that a large amount of memory space is consumed by index data is not eliminated; for the vector Quantization method, the representative algorithm is a Product Quantization (PQ) method which is already very practical and popular in the industrial field, the index data compression performance is better, the memory occupation can be effectively reduced, but the recall rate is lower.
In addition, in recent years, many cases employ FALCONN or NMSLIB frameworks that are not currently supported by dynamic addition and deletion of data in the index database of the CBIR method based on the ANN search policy. This is acceptable for search algorithms and system implementations for small data sets and medium-scale data sets. However, for large-scale retrieval, in order to meet the needs of practical application occasions, the CBIR system needs to perform storage of indexes on specific sensitive picture data in the operating process. Otherwise, each time the index library is re-established, high operation and maintenance cost and time consumption are required. Therefore, the index library for the CBIR system in practical applications should be dynamically scalable.
Disclosure of Invention
In view of the above, the present invention aims to provide a CBIR method based on the modified PQ algorithm.
In order to achieve the purpose, the invention provides the following technical scheme:
a content-based image retrieval CBIR method based on an improved product quantization PQ algorithm extracts image depth features through an improved depth convolution network, then codes compressed image feature data through an index retrieval module based on an inverted index product quantization IVPQ algorithm and adopting a nonlinear retrieval ANN search strategy, generates indexes of a dynamic index database based on a Faiss frame, segments a data space of a full index database through feature vector coding, rapidly locks a certain subspace through Hamming distance rearrangement for traversal when retrieval of a query image is carried out, and outputs a retrieval image.
Optionally, the IVPQ algorithm is divided into index construction and nonlinear search query, and X ═ X is recorded1,x2,...,xN]∈RN×ΩIs a characteristic vector data set matrix of a training sample set, wherein omega is the dimension of training sample data, N is the number of samples of the training sample set, and a query sample is xq；
The index construction specifically comprises the following steps:
and (3) carrying out encoding preprocessing: performing a K-Means clustering algorithm on the training sample characteristic vector data set X to obtain M sample clustering centers C ═ C1,c2,...,cM]∈RM×ΩIs provided with ci＝NNC(xi) Representing a training sample data feature vector xiIn the nearest sample clustering center, subtracting each two to obtain a residual vector group R, wherein the R is expressed by the formula:
R＝[r1,r2,...,ri,...,rN]∈RN×Ω
ri＝|xi-ci| (2)
for residual vector riThe dimension of omega is divided equally by P, and ri is [ r ]i,1,ri,2,...,ri,j,...,ri,P]∈R1 ×ΩAnd omega1+...+ωj+...+ωPAnd respectively carrying out K-Means clustering on residual sub-vectors of all training samples in different subspaces to generate a codebook set C with consistent clustering center numberΩ，CΩThe expression is as follows:
Figure BDA0002351547950000021
Figure BDA0002351547950000022
wherein the content of the first and second substances,
Figure BDA0002351547950000023
the method comprises the steps that a codebook (cluster set) of the jth dimensionality subspace formed after the dimensionality space omega of a training sample residual vector group R is divided equally is provided, and P is the number of the dimensionality subspaces divided equally by omega;
Figure BDA0002351547950000024
is composed of
Figure BDA0002351547950000025
M 'is the number of cluster centers per subspace, and satisfies M' 2p，2pBinary encoding the number of bits for IVPQ;
by CΩTo riIVPQ encoding with a per sample residual vector riExpressed by the ID number of the clustering center corresponding to the P residual sub-vectors, a training sample IVPQ coding set S is generated, wherein the S is expressed by the following formula:
S＝{S(1),S(2),...,S(i),...,S(N)}
Figure BDA0002351547950000031
Figure BDA0002351547950000032
wherein S (i) is a residual vector r of the training sampleiGenerated set of IVPQ codes, ciMarking the corresponding training sample cluster center; n (i, j) is the sample residual sub-vector r in S (i)i,jIn the corresponding dimension subspace ωjThe number of the nearest cluster center;
Figure BDA0002351547950000033
representing a subspace ωjNeutron vector ri,jThe nearest cluster center number;
the nonlinear retrieval query specifically comprises:
for query sample vector xqPerforming the similar coding preprocessing to generate a query residual vector rq＝|xq-cqL, likewise will rqDividing into P identical sub-vectors, and recording rq＝[rq,1,rq,2,...,rq,j,...,rq,P]∈R1*ΩAnd respectively calculating the distance between each subspace and M' clustering centers in the subspace to generate a query vector distance pool D with the size of P multiplied by MΩ，DΩThe expression is as follows:
Figure BDA0002351547950000034
Figure BDA0002351547950000035
wherein, cqTo query the sample cluster centers of the sample vectors,
Figure BDA0002351547950000036
for querying residual subvectors rq(j) And subspace omegajDistance sets of M' cluster centers;
Figure BDA0002351547950000037
is rq(j) Corresponding to omegajThe distance value of the k-th cluster center in (c),
Figure BDA0002351547950000038
is rq(j) Corresponding to omegajThe kth cluster center;
when searching, only the training sample coding set S and the query sample vector xqSample cluster center cqIVPQ code set S with consistent subscriptqNamely, ROI, traversing and inquiring; let the number of code groups consistent with the query vector be N', and obtain S from equation (4)qExpression:
Sq＝{Sq(1),Sq(2),...,Sq(i)...,Sq(N')}
Figure BDA0002351547950000039
Figure BDA00023515479500000310
in the query vector distance pool DΩRespectively calculate and SqGenerating a query search distance set D by the sum of P Hamming distance values corresponding to each coding groupqThen D isqThe expression is as follows:
Dq＝[Dq(1),Dq(2),...,Dq(i),...,Dq(N')]
Figure BDA0002351547950000041
wherein D isq(i) Denotes SqThe ith training sample vector xiAnd query sample vector xqIVPQ coding distance of; if the sum of distances Dq(i) Exceeding a threshold distance t, t e [30,100 ] set according to actual training needs]If yes, discarding; finally, each training sample and each inquiry sample are usedIs returned as a result of the non-linear search.
The invention has the beneficial effects that:
in terms of requirements, the CBIR method based on the IVPQ algorithm provided by the invention is suitable for being used in an actual application scene, and a user needs to search and judge a key-sensitive image library based on massive, label-free and complex unknown images, so that the function of searching images by images is realized; the dynamic index database is realized based on the Faiss framework, and the high operation and maintenance cost generated for reconstructing the index database in practical application occasions is avoided.
In the algorithm effect, the Product Quantization (PQ) algorithm of index coding is improved and optimized, and an optimization process of the product quantization (IVPQ) algorithm based on the inverted index is provided, so that the high-efficiency compression of data characteristics and good nonlinear retrieval are realized; the distance calculation times of the original PQ coding algorithm are times, while the distance calculation times of the IVPQ coding algorithm are only needed during query and retrieval, so that the calculation amount is greatly reduced, and the time consumption of the algorithm is optimized; because the retrieved test samples are clustered in advance, the retrieval recall rate can be improved to a certain extent in precision.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the means of the instrumentalities and combinations particularly pointed out hereinafter.
Drawings
For purposes of promoting a better understanding of the objects, aspects and advantages of the invention, reference will now be made to the following detailed description taken in conjunction with the accompanying drawings in which:
FIG. 1 is a flow chart of a CBIR method for improving IVPQ algorithm;
fig. 2 is the IVPQ algorithm process.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention in a schematic way, and the features in the following embodiments and examples may be combined with each other without conflict.
Wherein the showings are for the purpose of illustrating the invention only and not for the purpose of limiting the same, and in which there is shown by way of illustration only and not in the drawings in which there is no intention to limit the invention thereto; to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The same or similar reference numerals in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if there is an orientation or positional relationship indicated by terms such as "upper", "lower", "left", "right", "front", "rear", etc., based on the orientation or positional relationship shown in the drawings, it is only for convenience of description and simplification of description, but it is not an indication or suggestion that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes, and are not to be construed as limiting the present invention, and the specific meaning of the terms may be understood by those skilled in the art according to specific situations.
The invention provides a CBIR method based on an inverted product quantization (IVPQ) algorithm, which is used for realizing a large-scale image retrieval system based on a feature vector coding index. The specific process is as shown in fig. 1, an image depth feature is extracted by improving a depth convolution network, inverted product quantization coding compression features are performed on the image feature, an index of a dynamic index database based on a Faiss framework is generated, a data space of a full index database is divided through feature vector coding, when a query picture is retrieved, a certain subspace is quickly locked through hamming distance rearrangement and then traversed, and a retrieval image is output.
The IVPQ algorithm is divided into two steps of index construction and nonlinear retrieval query, and the specific flow is shown in FIG. 2.
Let X be ═ X1,x2,...,xN]∈RN×ΩA characteristic vector data set matrix of a training sample set, wherein omega is the dimension of training sample data, N is the number of samples of the training sample set, and a query sample is xq。
The index construction steps of IVPQ are as follows:
and carrying out coding preprocessing. Performing a K-Means clustering algorithm on the training sample characteristic vector data set X to obtain M sample clustering centers C ═ C1,c2,...,cM]∈RM×ΩIs provided with ci＝NNC(xi) Representing a training sample data feature vector xiIn the nearest sample clustering center, subtracting each two to obtain a residual vector group R, wherein the R is expressed by the formula:
R＝[r1,r2,...,ri,...,rN]∈RN×Ω
ri＝|xi-ci| (2)
for residual vector riThe dimension space omega of (A) is divided equally by P, and r is recordedi＝[ri,1,ri,2,...,ri,j,...,ri,P]∈R1 ×ΩAnd omega1+...+ωj+...+ωPAnd respectively carrying out K-Means clustering on residual sub-vectors of all training samples in different subspaces to generate a codebook set C with consistent clustering center numberΩ，CΩThe expression is as follows:
Figure BDA0002351547950000061
Figure BDA0002351547950000062
wherein the content of the first and second substances,
Figure BDA0002351547950000063
the method comprises the steps that a codebook (cluster set) of the jth dimensionality subspace formed after the dimensionality space omega of a training sample residual vector group R is divided equally is provided, and P is the number of the dimensionality subspaces divided equally by omega;
Figure BDA0002351547950000064
is composed of
Figure BDA0002351547950000065
M 'is the number of cluster centers per subspace, and satisfies M' 2p，2pThe number of bits is binary coded for IVPQ.
By CΩTo riIVPQ encoding with a per sample residual vector riExpressed by the ID number of the clustering center corresponding to the P residual sub-vectors, a training sample IVPQ coding set S is generated, wherein the S is expressed by the following formula:
S＝{S(1),S(2),...,S(i),...,S(N)}
Figure BDA0002351547950000066
Figure BDA0002351547950000067
wherein S (i) is a residual vector r of the training sampleiGenerated set of IVPQ codes, ciMarking the corresponding training sample cluster center; n (i, j) is the sample residual sub-vector r in S (i)i,jIn the corresponding dimension subspace ωjThe number of the nearest cluster center;
Figure BDA0002351547950000068
representing a subspace ωjNeutron vector ri,jThe nearest cluster center number.
The non-linear retrieval procedure for IVPQ is as follows:
for query sample vector xqPerforming the sameCode pre-processing to generate query residual vector rq＝|xq-cqL, likewise will rqDividing into P identical sub-vectors, and recording rq＝[rq,1,rq,2,...,rq,j,...,rq,P]∈R1*ΩAnd respectively calculating the distance between each subspace and M' clustering centers in the subspace to generate a query vector distance pool D with the size of P multiplied by MΩ，DΩThe expression is as follows:
Figure BDA0002351547950000069
Figure BDA00023515479500000610
wherein, cqTo query the sample cluster centers of the sample vectors,
Figure BDA00023515479500000611
for querying residual subvectors rq(j) And subspace omegajDistance sets of M' cluster centers;
Figure BDA00023515479500000612
is rq(j) Corresponding to omegajThe distance value of the k-th cluster center in (c),
Figure BDA00023515479500000613
is rq(j) Corresponding to omegajThe k-th cluster center.
When searching, only the training sample coding set S and the query sample vector xqSample cluster center cqIVPQ code set S with consistent subscriptq(region of interest ROI) is traversed. Let the number of code groups consistent with the query vector be N', and S can be obtained from equation (4)qExpression:
Sq＝{Sq(1),Sq(2),...,Sq(i)...,Sq(N')}
Figure BDA0002351547950000071
Figure BDA0002351547950000072
in the query vector distance pool DΩRespectively calculate and SqGenerating a query search distance set D by the sum of P Hamming distance values corresponding to each coding groupqThen D isqThe expression is as follows:
Dq＝[Dq(1),Dq(2),...,Dq(i),...,Dq(N')]
Figure BDA0002351547950000073
wherein D isq(i) Denotes SqThe ith training sample vector xiAnd query sample vector xqIVPQ coding distance. If the sum of distances Dq(i) Exceeding a threshold distance t, t e [30,100 ] set according to actual training needs]It can be discarded. And finally, sorting the distance between each training sample and the query sample as a result of the nonlinear retrieval and returning.
The invention provides a CBIR method for improving product quantization coding, which comprises the following specific processes: adopting an improved residual error network as an image feature extractor; and then coding compressed image characteristic data by an index retrieval module of an IVPQ coding algorithm adopting a nonlinear retrieval ANN search strategy, generating an index in a dynamic index database based on a Faiss frame, and finally performing retrieval judgment by utilizing Hamming distance measurement.
Meanwhile, two embodiments created by the invention are given:
1. and (3) carrying out an index algorithm test:
the SIFT1M is used as a test data set, the IVPQ coding retrieval algorithm improved and optimized by the method is compared with the existing image retrieval algorithms based on ANN retrieval strategies, and the superiority of the retrieval algorithm is measured by three indexes, namely recall rate, retrieval time and index file size. The graph proposed by the literature is introduced into the index quantization method HNSW, respectively. The experimental parameters are described as follows: nlist: representing the number of sample clusters; m: representing the number of the divided subspaces; nbit: a binary encoding number representing each vector quantum space; nprobe represents the number of the most similar classes searched during query; r @ n represents the recall rate for returning n most similar IDs; time: representing the time required for a single query vector to complete retrieval; the file size is as follows: and the size of the memory space occupied by the generated index file is represented.
TABLE 1 IVPQ Performance test
Figure BDA0002351547950000081
TABLE 2 PQ Performance test
Figure BDA0002351547950000082
TABLE 3 MLSH Performance test
Figure BDA0002351547950000083
Figure BDA0002351547950000091
TABLE 4 HNSW Performance test
Figure BDA0002351547950000092
TABLE 5 violence retrieval test (ground-truth)
Figure BDA0002351547950000093
The results of the tests are shown in tables 1, 2, 3, 4, and 5: and (4) searching by brute-force (brute-force) of cosine distance (the normalized cosine similarity is equivalent to Euclidean L2 distance) to be used as a data reference standard true value (ground-true). The test result shows that the IVPQ retrieval requires much less time than PQ and MLSH compared with HNSW as the value of m increases under the condition of the same number of coded bits, and IVPQ coding is only slightly lower than PQ coding but better than LSH coding and HNSW coding in the recall ratio. Because the IVPQ coding requires the storage of the sample cluster center, the index file generated by the optimized and improved algorithm is necessarily slightly larger than that generated by the prototype PQ algorithm. Under the conditions of not much recall rate and the same number of encoding bits, the index file generated by IVPQ encoding is only slightly higher than PQ, but is far smaller than the index files of LSH and HNSW, and the compression efficiency is respectively improved by 62.24% and 76.52% compared with the index files of LSH and HNSW; the retrieval method of LSH, HNSW and PQ consumes much less time for IVPQ retrieval, the retrieval time is only 4.03%, 13.21% and 27.05% of HNSW, LSH and PQ respectively, and the speed is improved by 23.21 times, 6.56 times and 2.69 times. By comprehensive evaluation, the index module adopting the IVPQ retrieval algorithm is most superior in performance under the condition of facing a large-scale data set.
2. The retrieval effect display test of the CBIR method is carried out by using a Caltech256 image data set as a test data set, a feature extractor of the CBIR method respectively adopts the improved residual network model ResNet152v2_ AEPL, and an index retrieval module respectively adopts the IVPQ algorithm proposed by the method and the index retrieval algorithm compared with other documents HNSW, MLSH and PQ:
the CBIR method proposed by the invention extracts the depth feature descriptors of the Caltech256 image dataset by means of a feature extractor. And the index retrieval module is used for encoding and compressing the feature data to generate 28780 index sample index libraries and 1000 query indexes of test samples, and the performance test of the CBIR retrieval system is carried out. The results of this experiment are shown for elk (Caltech256-065.elk) search, respectively. The test result further verifies the retrieval effect precision of the CBIR method based on the IVPQ algorithm.
Finally, the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions, and all that should be covered by the claims of the present invention.

Claims (2)

1. A content-based image retrieval CBIR method based on an improved product quantization PQ algorithm is characterized in that: the method comprises the steps of extracting image depth features through an improved depth convolution network, coding and compressing image feature data through an index retrieval module of an inverted index based product quantification IVPQ algorithm which adopts a nonlinear retrieval ANN search strategy, generating indexes of a dynamic index database based on a Faiss frame, partitioning a data space of a full index database through feature vector coding, rapidly locking a certain subspace through Hamming distance rearrangement for traversal when retrieval of a query picture is carried out, and outputting a retrieval image.
2. The CBIR method based on modified PQ algorithm of claim 1, wherein: the IVPQ algorithm is divided into index construction and nonlinear retrieval query, and the notation of X is [ X ]1,x2,...,xN]∈RN×ΩA characteristic vector data set matrix of a training sample set, wherein omega is the dimension of training sample data, N is the number of samples of the training sample set, and a query sample is xq；
The index construction specifically comprises the following steps:
and (3) carrying out encoding preprocessing: performing a K-Means clustering algorithm on the training sample characteristic vector data set X to obtain M sample clustering centers C ═ C1,c2,...,cM]∈RM×ΩIs provided with ci＝NNC(xi) Representing a training sample data feature vector xiAnd (3) subtracting the nearest sample clustering center by two to obtain a residual vector group R, wherein the R is expressed by the formula:
R＝[r1,r2,...,ri,...,rN]∈RN×Ω
ri＝|xi-ci| (2)
for residual vector riThe dimension of omega is divided into P halves,note ri＝[ri,1,ri,2,...,ri,j,...,ri,P]∈R1×ΩAnd omega1+...+ωj+...+ωPAnd respectively carrying out K-Means clustering on residual sub-vectors of all training samples in different subspaces to generate a codebook set C with consistent clustering center numberΩ，CΩThe expression is as follows:
Figure FDA0002351547940000011
Figure FDA0002351547940000012
wherein the content of the first and second substances,
Figure FDA0002351547940000013
a codebook (cluster set) of the jth dimensionality subspace formed by halving the dimensionality space omega of the training sample residual vector group R is provided, and P is the number of the dimensionality subspaces after the omega is halved;
Figure FDA0002351547940000014
is composed of
Figure FDA0002351547940000015
M 'is the number of cluster centers per subspace, and satisfies M' 2p，2pBinary encoding the number of bits for IVPQ;
by CΩTo riIVPQ encoding with a per sample residual vector riExpressed by the ID number of the clustering center corresponding to the P residual sub-vectors, a training sample IVPQ coding set S is generated, wherein the S is expressed by the following formula:
S＝{S(1),S(2),...,S(i),...,S(N)}
Figure FDA0002351547940000016
Figure FDA0002351547940000021
wherein S (i) is a residual vector r of the training sampleiGenerated set of IVPQ codes, ciMarking the corresponding training sample cluster center; n (i, j) is the sample residual sub-vector r in S (i)i,jIn the corresponding dimension subspace ωjThe number of the nearest cluster center;
Figure FDA0002351547940000022
representing a subspace ωjNeutron vector ri,jThe nearest cluster center number;
the nonlinear retrieval query specifically:
for query sample vector xqPerforming the similar coding preprocessing to generate a query residual vector rq＝|xq-cqL, likewise will rqDividing into P identical sub-vectors, and recording rq＝[rq,1,rq,2,...,rq,j,...,rq,P]∈R1*ΩAnd respectively calculating the distance between each subspace and M' clustering centers in the subspace to generate a query vector distance pool D with the size of P multiplied by MΩ，DΩThe expression is as follows:
Figure FDA0002351547940000023
Figure FDA0002351547940000024
wherein, cqTo query the sample cluster centers of the sample vectors,
Figure FDA0002351547940000025
for querying residual subvectors rq(j) And subspace omegajDistance sets of M' cluster centers;
Figure FDA0002351547940000026
is rq(j) Corresponding to omegajThe distance value of the k-th cluster center in (c),
Figure FDA0002351547940000027
is rq(j) Corresponding to omegajThe kth cluster center;
when searching, only the training sample coding set S and the query sample vector xqSample cluster center cqIVPQ code set S with consistent subscriptqNamely, ROI, traversing and inquiring; let the number of code groups consistent with the query vector be N', and obtain S from equation (4)qExpression:
Sq＝{Sq(1),Sq(2),...,Sq(i)...,Sq(N')}
Figure FDA0002351547940000028
Figure FDA0002351547940000029
in the query vector distance pool DΩRespectively calculate and SqGenerating a query retrieval distance set D by the sum of P Hamming distance values corresponding to each coding groupqThen D isqThe expression is as follows:
Dq＝[Dq(1),Dq(2),...,Dq(i),...,Dq(N')]
Figure FDA00023515479400000210
wherein D isq(i) Denotes SqThe ith training sample vector xiAnd query sample vector xqIVPQ coding distance of; if the sum of the distances Dq(i) Exceeding a threshold distance t, t e [30,100 ] set according to actual training needs]If yes, discarding; finally, the distance between each training sample and the query sample is used for sortingAnd returning the result of the nonlinear retrieval.
CN201911417377.6A 2019-12-31 2019-12-31 CBIR method based on improved PQ algorithm Active CN111177435B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911417377.6A CN111177435B (en) 2019-12-31 2019-12-31 CBIR method based on improved PQ algorithm

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911417377.6A CN111177435B (en) 2019-12-31 2019-12-31 CBIR method based on improved PQ algorithm

Publications (2)

Publication Number Publication Date
CN111177435A true CN111177435A (en) 2020-05-19
CN111177435B CN111177435B (en) 2023-03-31

Family

ID=70650651

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911417377.6A Active CN111177435B (en) 2019-12-31 2019-12-31 CBIR method based on improved PQ algorithm

Country Status (1)

Country Link
CN (1) CN111177435B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111797259A (en) * 2020-07-10 2020-10-20 杭州慕锐科技有限公司 Rapid image searching method for wrinkled deformation
CN114595350A (en) * 2021-12-08 2022-06-07 拓尔思信息技术股份有限公司 Method for quickly searching billion-level images
CN115599791A (en) * 2022-11-15 2023-01-13 以萨技术股份有限公司（Cn） Milvus database parameter determination method, device and storage medium
CN116595233A (en) * 2023-06-02 2023-08-15 上海爱可生信息技术股份有限公司 Vector database retrieval processing acceleration method and system based on NPU

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150169644A1 (en) * 2013-01-03 2015-06-18 Google Inc. Shape-Gain Sketches for Fast Image Similarity Search
CN106844524A (en) * 2016-12-29 2017-06-13 北京工业大学 A kind of medical image search method converted based on deep learning and Radon
US20180341805A1 (en) * 2015-11-06 2018-11-29 Thomson Licensing Method and Apparatus for Generating Codebooks for Efficient Search
CN110472091A (en) * 2019-08-22 2019-11-19 深圳市商汤科技有限公司 Image processing method and device, electronic equipment and storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150169644A1 (en) * 2013-01-03 2015-06-18 Google Inc. Shape-Gain Sketches for Fast Image Similarity Search
US20180341805A1 (en) * 2015-11-06 2018-11-29 Thomson Licensing Method and Apparatus for Generating Codebooks for Efficient Search
CN106844524A (en) * 2016-12-29 2017-06-13 北京工业大学 A kind of medical image search method converted based on deep learning and Radon
CN110472091A (en) * 2019-08-22 2019-11-19 深圳市商汤科技有限公司 Image processing method and device, electronic equipment and storage medium

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
BENJAMIN KLEIN ET AL.: "End-to-End Supervised Product Quantization for Image Search and Retrieval" *
GIUSPEPPE AMATO ET AL.: "large-scale instance-level image retrieval" *
MENG ZHAO: "An angle structure description for image retrieval" *
ZHENLUAN XUE ET AL.: "Non-invasive through-skull brain vascular imaging and small tumor diagnosis based on NIR-II emissive lanthanide nanoprobes beyond 1500 nm" *
杜丹蕾: "面向图像检索的累加乘积量化方法研究" *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111797259A (en) * 2020-07-10 2020-10-20 杭州慕锐科技有限公司 Rapid image searching method for wrinkled deformation
CN114595350A (en) * 2021-12-08 2022-06-07 拓尔思信息技术股份有限公司 Method for quickly searching billion-level images
CN114595350B (en) * 2021-12-08 2024-04-26 拓尔思信息技术股份有限公司 Billion-level image quick searching method
CN115599791A (en) * 2022-11-15 2023-01-13 以萨技术股份有限公司（Cn） Milvus database parameter determination method, device and storage medium
CN115599791B (en) * 2022-11-15 2023-03-10 以萨技术股份有限公司 Milvus database parameter determination method, device and storage medium
CN116595233A (en) * 2023-06-02 2023-08-15 上海爱可生信息技术股份有限公司 Vector database retrieval processing acceleration method and system based on NPU

Also Published As

Publication number Publication date
CN111177435B (en) 2023-03-31

Similar Documents

Publication Publication Date Title
CN111177435B (en) CBIR method based on improved PQ algorithm
Wu et al. Multiscale quantization for fast similarity search
Norouzi et al. Fast exact search in hamming space with multi-index hashing
Zhou et al. Scalar quantization for large scale image search
US7725484B2 (en) Scalable object recognition using hierarchical quantization with a vocabulary tree
Song et al. Robust hashing with local models for approximate similarity search
CN108280187B (en) Hierarchical image retrieval method based on depth features of convolutional neural network
Liao et al. A sample-based hierarchical adaptive K-means clustering method for large-scale video retrieval
CN106033426A (en) A latent semantic min-Hash-based image retrieval method
CN108491430A (en) It is a kind of based on the unsupervised Hash search method clustered to characteristic direction
Ferdowsi et al. Fast content identification in high-dimensional feature spaces using sparse ternary codes
Liu et al. An image-based near-duplicate video retrieval and localization using improved edit distance
Negrel et al. Web-scale image retrieval using compact tensor aggregation of visual descriptors
Wang et al. Statistical quantization for similarity search
CN110046660A (en) A kind of product quantization method based on semi-supervised learning
Zhang et al. Effective image retrieval via multilinear multi-index fusion
Yu et al. Scalable video event retrieval by visual state binary embedding
Liu et al. TOP-SIFT: the selected SIFT descriptor based on dictionary learning
Dong et al. Holons visual representation for image retrieval
Wang et al. Lider: An efficient high-dimensional learned index for large-scale dense passage retrieval
Cheng et al. Comparative study on dimensionality reduction in large-scale image retrieval
Mu et al. Randomized locality sensitive vocabularies for bag-of-features model
Sun et al. Search by detection: Object-level feature for image retrieval
Kulkarni et al. An effective content based video analysis and retrieval using pattern indexing techniques
Zhang et al. Modeling spatial and semantic cues for large-scale near-duplicated image retrieval

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant